2024-09-11T01:02:21.525 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T01:02:21.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T01:02:21.637 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880 branch: wip-hemanth-testing-2024-09-10-0723 description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-lz4} rados supported-random-distro$/{centos_latest}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899880' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(MON_DOWN\) sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 9167 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 targets: smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDi3saszkDpcglMo1+hBVtJhZXO0hCIWGWZzsgbgqJg6rJ7DGWjLKVbqPXCUiRfLLHAjpjcIYEeet20R/yEUClM= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - lost_unfound: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:16:14 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T01:02:21.637 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-11T01:02:21.638 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-11T01:02:21.638 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T01:02:21.640 INFO:teuthology.task.internal:Checking packages... 2024-09-11T01:02:21.664 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-11T01:02:21.665 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T01:02:21.665 INFO:teuthology.packaging:ref: None 2024-09-11T01:02:21.665 INFO:teuthology.packaging:tag: None 2024-09-11T01:02:21.665 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-11T01:02:21.665 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T01:02:21.665 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-11T01:02:21.826 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-11T01:02:21.828 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T01:02:21.837 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T01:02:21.837 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T01:02:21.875 INFO:teuthology.task.internal:Saving configuration 2024-09-11T01:02:21.887 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T01:02:21.895 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T01:02:21.928 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 00:56:32.802418', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDi3saszkDpcglMo1+hBVtJhZXO0hCIWGWZzsgbgqJg6rJ7DGWjLKVbqPXCUiRfLLHAjpjcIYEeet20R/yEUClM='} 2024-09-11T01:02:21.929 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T01:02:21.935 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-11T01:02:21.935 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T01:02:21.980 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd09e47bbe0>, signals=[15]) 2024-09-11T01:02:21.980 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T01:02:21.985 INFO:teuthology.task.internal:Opening connections... 2024-09-11T01:02:21.986 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:02:21.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:02:22.078 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T01:02:22.085 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2024-09-11T01:02:22.159 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2024-09-11T01:02:22.159 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2024-09-11T01:02:22.217 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="CentOS Stream" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="9" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:ID="centos" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE="rhel fedora" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="9" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:PLATFORM_ID="platform:el9" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:ANSI_COLOR="0;31" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:LOGO="fedora-logo-icon" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://centos.org/" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-11T01:02:22.218 INFO:teuthology.orchestra.run.smithi039.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-11T01:02:22.219 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2024-09-11T01:02:22.247 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T01:02:22.258 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T01:02:22.264 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T01:02:22.264 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2024-09-11T01:02:22.283 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T01:02:22.289 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T01:02:22.290 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2024-09-11T01:02:22.352 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T01:02:22.388 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T01:02:22.402 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T01:02:22.403 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T01:02:22.403 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T01:02:22.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:02:22.420 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-11T01:02:22.478 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-503.el9.x86_64 2024-09-11T01:02:22.478 INFO:teuthology.task.kernel:Running kernel on smithi039: 5.14.0-503.el9.x86_64 2024-09-11T01:02:22.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum install -y kernel 2024-09-11T01:02:26.115 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.2 MB 00:01 2024-09-11T01:02:30.082 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 20 MB 00:02 2024-09-11T01:02:34.558 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-11T01:02:36.125 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-11T01:02:36.938 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 33 MB/s | 23 MB 00:00 2024-09-11T01:02:42.668 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-11T01:02:44.783 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-11T01:02:44.783 INFO:teuthology.orchestra.run.smithi039.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-11T01:02:44.870 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:02:44.874 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:02:44.874 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-09-11T01:02:44.874 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:02:44.874 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:02:44.874 INFO:teuthology.orchestra.run.smithi039.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout:Install 4 Packages 2024-09-11T01:02:44.875 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:02:44.876 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 79 M 2024-09-11T01:02:44.876 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 126 M 2024-09-11T01:02:44.876 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:02:45.095 INFO:teuthology.orchestra.run.smithi039.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 245 kB/s | 23 kB 00:00 2024-09-11T01:02:47.611 INFO:teuthology.orchestra.run.smithi039.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.0 MB/s | 16 MB 00:02 2024-09-11T01:02:50.511 INFO:teuthology.orchestra.run.smithi039.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 5.3 MB/s | 29 MB 00:05 2024-09-11T01:02:50.920 INFO:teuthology.orchestra.run.smithi039.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.8 MB/s | 35 MB 00:05 2024-09-11T01:02:50.920 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:02:50.920 INFO:teuthology.orchestra.run.smithi039.stdout:Total 13 MB/s | 79 MB 00:06 2024-09-11T01:02:51.627 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:02:51.999 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:02:52.000 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:02:52.991 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:02:52.991 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:02:55.564 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:02:56.249 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-11T01:02:56.328 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T01:02:59.134 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T01:02:59.267 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T01:03:04.386 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T01:03:04.504 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T01:03:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T01:03:48.996 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T01:03:49.038 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-11T01:03:49.370 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-11T01:03:49.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-11T01:03:49.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-11T01:03:49.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-11T01:03:50.225 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-11T01:03:50.225 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:03:50.225 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:03:50.225 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-11T01:03:50.225 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-11T01:03:50.226 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-11T01:03:50.226 INFO:teuthology.orchestra.run.smithi039.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-11T01:03:50.226 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:03:50.226 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:03:50.580 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T01:03:50.670 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T01:03:50.672 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T01:03:50.672 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-11T01:03:50.673 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T01:03:50.673 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T01:03:50.673 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T01:03:50.673 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi039.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T01:03:50.673 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q kernel | sort -rV | head -n 1 2024-09-11T01:03:50.731 INFO:teuthology.orchestra.run.smithi039.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-11T01:03:50.732 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-11T01:03:50.732 DEBUG:teuthology.orchestra.run.smithi039:> sudo rpm -qi grub2-tools 2024-09-11T01:03:50.765 INFO:teuthology.orchestra.run.smithi039.stdout:Name : grub2-tools 2024-09-11T01:03:50.765 INFO:teuthology.orchestra.run.smithi039.stdout:Epoch : 1 2024-09-11T01:03:50.765 INFO:teuthology.orchestra.run.smithi039.stdout:Version : 2.06 2024-09-11T01:03:50.765 INFO:teuthology.orchestra.run.smithi039.stdout:Release : 82.el9 2024-09-11T01:03:50.765 INFO:teuthology.orchestra.run.smithi039.stdout:Architecture: x86_64 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Group : Unspecified 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Size : 8274670 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:License : GPLv3+ 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Packager : builder@centos.org 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Vendor : CentOS 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Summary : Support tools for GRUB. 2024-09-11T01:03:50.766 INFO:teuthology.orchestra.run.smithi039.stdout:Description : 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout:hardware devices. 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:03:50.767 INFO:teuthology.orchestra.run.smithi039.stdout:This subpackage provides tools for support of all platforms. 2024-09-11T01:03:50.768 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-11T01:03:50.768 INFO:teuthology.task.kernel:Updating grub on smithi039 to boot 5.14.0-505.el9.x86_64 2024-09-11T01:03:50.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-11T01:03:51.435 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2024-09-11T01:03:52.753 INFO:teuthology.orchestra.run.smithi039.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-11T01:03:52.782 INFO:teuthology.orchestra.run.smithi039.stderr:done 2024-09-11T01:03:52.790 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-11T01:03:52.805 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.5FeSjlJEYw 2024-09-11T01:03:52.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.5FeSjlJEYw 2024-09-11T01:03:52.872 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.5FeSjlJEYw 2024-09-11T01:03:53.008 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.5FeSjlJEYw is 6KB 2024-09-11T01:03:53.055 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.5FeSjlJEYw 2024-09-11T01:03:53.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo /bin/ls /boot/loader/entries || true 2024-09-11T01:03:53.132 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-11T01:03:53.132 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-11T01:03:53.132 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-11T01:03:53.132 INFO:teuthology.orchestra.run.smithi039.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-11T01:03:53.133 DEBUG:teuthology.orchestra.run.smithi039:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-11T01:03:53.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo shutdown -r now 2024-09-11T01:04:23.321 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T01:04:23.322 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:04:23.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:04:57.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-11T01:05:06.926 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:05:06.927 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:05:10.017 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2024-09-11T01:05:22.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:05:22.029 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:06:22.101 DEBUG:teuthology.orchestra.remote:timed out 2024-09-11T01:06:37.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:06:37.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:06:37.450 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-11T01:06:37.809 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:06:37.809 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-11T01:06:37.809 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2024-09-11T01:06:37.826 INFO:teuthology.orchestra.run.smithi039.stdout:5.14.0-505.el9.x86_64 2024-09-11T01:06:37.826 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-11T01:06:37.826 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T01:06:37.826 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-11T01:06:38.827 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T01:06:38.827 DEBUG:teuthology.orchestra.run.smithi039:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T01:06:38.906 INFO:teuthology.orchestra.run.smithi039.stdout:ttyS1 2024-09-11T01:06:38.931 DEBUG:teuthology.parallel:result is None 2024-09-11T01:06:38.931 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T01:06:38.939 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T01:06:38.939 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T01:06:38.959 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T01:06:38.964 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T01:06:38.973 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T01:06:38.973 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T01:06:39.023 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T01:06:39.030 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T01:06:39.030 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T01:06:39.102 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T01:06:39.114 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T01:06:39.115 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T01:06:39.123 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T01:06:39.124 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T01:06:39.183 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T01:06:39.192 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T01:06:39.192 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T01:06:39.241 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T01:06:39.334 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T01:06:39.375 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T01:06:39.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T01:06:39.501 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:06:39.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T01:06:39.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2024-09-11T01:06:39.656 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T01:06:40.027 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T01:06:40.034 INFO:teuthology.task.internal:Starting timer... 2024-09-11T01:06:40.034 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T01:06:40.046 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T01:06:40.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo service auditd rotate 2024-09-11T01:06:40.117 INFO:teuthology.orchestra.run.smithi039.stdout:Rotating logs: 2024-09-11T01:06:40.119 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T01:06:40.120 DEBUG:teuthology.orchestra.run.smithi039:> /usr/sbin/getenforce 2024-09-11T01:06:40.152 INFO:teuthology.orchestra.run.smithi039.stdout:Permissive 2024-09-11T01:06:40.152 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi039.front.sepia.ceph.com': 'permissive'} 2024-09-11T01:06:40.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T01:06:40.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:06:40.219 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T01:06:40.220 DEBUG:teuthology.orchestra.run.smithi039:> sudo /usr/sbin/setenforce permissive 2024-09-11T01:06:40.291 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T01:06:40.300 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T01:06:40.317 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T01:06:40.318 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi039.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T01:10:28.622 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi039.front.sepia.ceph.com')] 2024-09-11T01:10:28.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:10:28.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:10:28.712 DEBUG:teuthology.orchestra.run.smithi039:> true 2024-09-11T01:10:28.797 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2024-09-11T01:10:28.797 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T01:10:28.807 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T01:10:28.808 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T01:10:28.808 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T01:10:28.867 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-11T01:10:28.881 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-11T01:10:28.906 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: ntpd: command not found 2024-09-11T01:10:28.918 INFO:teuthology.orchestra.run.smithi039.stdout:506 Cannot talk to daemon 2024-09-11T01:10:28.933 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-11T01:10:28.948 INFO:teuthology.orchestra.run.smithi039.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-11T01:10:29.007 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-09-11T01:10:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T01:10:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-09-11T01:10:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T01:10:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T01:10:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T01:10:29.011 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T01:10:29.012 INFO:teuthology.run_tasks:Running task install... 2024-09-11T01:10:29.019 DEBUG:teuthology.task.install:project ceph 2024-09-11T01:10:29.020 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-11T01:10:29.020 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-11T01:10:29.020 INFO:teuthology.task.install:Using flavor: default 2024-09-11T01:10:29.025 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T01:10:29.025 INFO:teuthology.task.install:extra packages: [] 2024-09-11T01:10:29.025 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-11T01:10:29.026 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T01:10:29.183 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-11T01:10:29.183 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-11T01:10:29.303 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-11T01:10:29.303 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:10:29.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-11T01:10:29.330 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-11T01:10:29.331 DEBUG:teuthology.orchestra.run.smithi039:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-11T01:10:29.398 DEBUG:teuthology.orchestra.run.smithi039:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-11T01:10:29.476 DEBUG:teuthology.orchestra.run.smithi039:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-11T01:10:29.541 INFO:teuthology.orchestra.run.smithi039.stdout:check_obsoletes = 1 2024-09-11T01:10:29.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum clean all 2024-09-11T01:10:29.779 INFO:teuthology.orchestra.run.smithi039.stdout:45 files removed 2024-09-11T01:10:29.805 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-radosgw 2024-09-11T01:10:30.392 INFO:teuthology.orchestra.run.smithi039.stdout:ceph packages for x86_64 292 kB/s | 84 kB 00:00 2024-09-11T01:10:30.685 INFO:teuthology.orchestra.run.smithi039.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-11T01:10:30.960 INFO:teuthology.orchestra.run.smithi039.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-11T01:10:32.060 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - BaseOS 7.8 MB/s | 8.2 MB 00:01 2024-09-11T01:10:34.903 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-11T01:10:40.112 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.5 MB 00:01 2024-09-11T01:10:41.788 INFO:teuthology.orchestra.run.smithi039.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 19 kB 00:00 2024-09-11T01:10:42.934 INFO:teuthology.orchestra.run.smithi039.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-11T01:10:48.607 INFO:teuthology.orchestra.run.smithi039.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2024-09-11T01:10:51.119 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:10:51.137 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:10:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:10:51.137 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:10:51.137 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:10:51.137 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-11T01:10:51.137 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrading: 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-11T01:10:51.138 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-11T01:10:51.139 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-11T01:10:51.140 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-11T01:10:51.141 INFO:teuthology.orchestra.run.smithi039.stdout:Installing weak dependencies: 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout:Install 31 Packages 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout:Upgrade 2 Packages 2024-09-11T01:10:51.142 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:10:51.143 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 62 M 2024-09-11T01:10:51.143 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:10:51.633 INFO:teuthology.orchestra.run.smithi039.stdout:(1/33): ceph-base-19.3.0-4854.g8f219c47.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-11T01:10:51.683 INFO:teuthology.orchestra.run.smithi039.stdout:(2/33): ceph-selinux-19.3.0-4854.g8f219c47.el9. 505 kB/s | 25 kB 00:00 2024-09-11T01:10:51.775 INFO:teuthology.orchestra.run.smithi039.stdout:(3/33): libcephfs2-19.3.0-4854.g8f219c47.el9.x8 8.1 MB/s | 755 kB 00:00 2024-09-11T01:10:51.859 INFO:teuthology.orchestra.run.smithi039.stdout:(4/33): libradosstriper1-19.3.0-4854.g8f219c47. 5.9 MB/s | 507 kB 00:00 2024-09-11T01:10:52.001 INFO:teuthology.orchestra.run.smithi039.stdout:(5/33): ceph-radosgw-19.3.0-4854.g8f219c47.el9. 14 MB/s | 11 MB 00:00 2024-09-11T01:10:52.043 INFO:teuthology.orchestra.run.smithi039.stdout:(6/33): python3-ceph-argparse-19.3.0-4854.g8f21 1.1 MB/s | 45 kB 00:00 2024-09-11T01:10:52.151 INFO:teuthology.orchestra.run.smithi039.stdout:(7/33): librgw2-19.3.0-4854.g8f219c47.el9.x86_6 19 MB/s | 5.5 MB 00:00 2024-09-11T01:10:52.185 INFO:teuthology.orchestra.run.smithi039.stdout:(8/33): python3-ceph-common-19.3.0-4854.g8f219c 1.0 MB/s | 143 kB 00:00 2024-09-11T01:10:52.210 INFO:teuthology.orchestra.run.smithi039.stdout:(9/33): python3-cephfs-19.3.0-4854.g8f219c47.el 2.7 MB/s | 161 kB 00:00 2024-09-11T01:10:52.252 INFO:teuthology.orchestra.run.smithi039.stdout:(10/33): python3-rados-19.3.0-4854.g8f219c47.el 4.7 MB/s | 321 kB 00:00 2024-09-11T01:10:52.277 INFO:teuthology.orchestra.run.smithi039.stdout:(11/33): python3-rbd-19.3.0-4854.g8f219c47.el9. 4.5 MB/s | 306 kB 00:00 2024-09-11T01:10:52.303 INFO:teuthology.orchestra.run.smithi039.stdout:(12/33): python3-rgw-19.3.0-4854.g8f219c47.el9. 1.9 MB/s | 99 kB 00:00 2024-09-11T01:10:52.562 INFO:teuthology.orchestra.run.smithi039.stdout:(13/33): ceph-common-19.3.0-4854.g8f219c47.el9. 16 MB/s | 22 MB 00:01 2024-09-11T01:10:52.903 INFO:teuthology.orchestra.run.smithi039.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 171 kB/s | 107 kB 00:00 2024-09-11T01:10:52.987 INFO:teuthology.orchestra.run.smithi039.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 249 kB/s | 171 kB 00:00 2024-09-11T01:10:53.054 INFO:teuthology.orchestra.run.smithi039.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 326 kB/s | 160 kB 00:00 2024-09-11T01:10:53.246 INFO:teuthology.orchestra.run.smithi039.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 132 kB/s | 45 kB 00:00 2024-09-11T01:10:53.571 INFO:teuthology.orchestra.run.smithi039.stdout:(18/33): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-09-11T01:10:53.596 INFO:teuthology.orchestra.run.smithi039.stdout:(19/33): python3-prettytable-0.7.2-27.el9.noarc 121 kB/s | 42 kB 00:00 2024-09-11T01:10:53.621 INFO:teuthology.orchestra.run.smithi039.stdout:(20/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 515 kB/s | 292 kB 00:00 2024-09-11T01:10:53.855 INFO:teuthology.orchestra.run.smithi039.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 139 kB/s | 36 kB 00:00 2024-09-11T01:10:53.914 INFO:teuthology.orchestra.run.smithi039.stdout:(22/33): gperftools-libs-2.9.1-2.el9.x86_64.rpm 5.2 MB/s | 309 kB 00:00 2024-09-11T01:10:53.948 INFO:teuthology.orchestra.run.smithi039.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 145 kB/s | 47 kB 00:00 2024-09-11T01:10:53.981 INFO:teuthology.orchestra.run.smithi039.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 788 kB/s | 26 kB 00:00 2024-09-11T01:10:53.998 INFO:teuthology.orchestra.run.smithi039.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.8 MB/s | 49 kB 00:00 2024-09-11T01:10:54.023 INFO:teuthology.orchestra.run.smithi039.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-11T01:10:54.074 INFO:teuthology.orchestra.run.smithi039.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 16 MB/s | 839 kB 00:00 2024-09-11T01:10:54.107 INFO:teuthology.orchestra.run.smithi039.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-11T01:10:54.191 INFO:teuthology.orchestra.run.smithi039.stdout:(29/33): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-11T01:10:54.216 INFO:teuthology.orchestra.run.smithi039.stdout:(30/33): qatlib-24.02.0-1.el9.x86_64.rpm 342 kB/s | 221 kB 00:00 2024-09-11T01:10:54.266 INFO:teuthology.orchestra.run.smithi039.stdout:(31/33): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-11T01:10:54.467 INFO:teuthology.orchestra.run.smithi039.stdout:(32/33): librados2-19.3.0-4854.g8f219c47.el9.x8 13 MB/s | 3.5 MB 00:00 2024-09-11T01:10:54.542 INFO:teuthology.orchestra.run.smithi039.stdout:(33/33): librbd1-19.3.0-4854.g8f219c47.el9.x86_ 10 MB/s | 3.2 MB 00:00 2024-09-11T01:10:54.546 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:10:54.546 INFO:teuthology.orchestra.run.smithi039.stdout:Total 18 MB/s | 62 MB 00:03 2024-09-11T01:10:54.668 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:10:54.710 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:10:54.710 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:10:55.419 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:10:55.419 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:10:56.984 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:10:57.077 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-11T01:10:57.106 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-11T01:10:57.139 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-11T01:10:57.170 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-11T01:10:57.192 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-11T01:10:57.390 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-11T01:10:57.453 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-11T01:10:57.812 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-11T01:10:57.924 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 7/35 2024-09-11T01:10:58.018 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 7/35 2024-09-11T01:10:58.108 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 8/35 2024-09-11T01:10:58.142 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/35 2024-09-11T01:10:58.205 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/35 2024-09-11T01:10:58.250 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-11T01:10:58.295 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9. 11/35 2024-09-11T01:10:58.366 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 12/35 2024-09-11T01:10:58.394 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_6 13/35 2024-09-11T01:10:58.458 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_6 13/35 2024-09-11T01:10:58.493 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-11T01:10:58.605 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-11T01:10:58.644 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-11T01:10:58.685 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-11T01:10:58.726 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/35 2024-09-11T01:10:58.755 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-11T01:10:58.796 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-11T01:10:58.842 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-11T01:10:59.176 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-11T01:10:59.218 INFO:teuthology.orchestra.run.smithi039.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 23/35 2024-09-11T01:10:59.310 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 23/35 2024-09-11T01:10:59.343 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 24/35 2024-09-11T01:10:59.393 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-11T01:10:59.926 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-11T01:10:59.953 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 27/35 2024-09-11T01:11:00.025 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 27/35 2024-09-11T01:11:00.106 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 28/35 2024-09-11T01:11:00.148 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x8 29/35 2024-09-11T01:11:02.584 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 30/35 2024-09-11T01:11:02.619 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 30/35 2024-09-11T01:11:03.245 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 30/35 2024-09-11T01:11:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 31/35 2024-09-11T01:11:03.381 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 31/35 2024-09-11T01:11:03.381 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-11T01:11:03.381 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-11T01:11:03.381 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:03.415 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 32/35 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 32/35 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /sys 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /proc 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /mnt 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /var/tmp 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /home 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /root 2024-09-11T01:11:13.886 INFO:teuthology.orchestra.run.smithi039.stdout:skipping the directory /tmp 2024-09-11T01:11:13.887 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:14.933 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 33/35 2024-09-11T01:11:14.960 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 33/35 2024-09-11T01:11:14.960 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:11:14.960 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-11T01:11:14.960 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T01:11:14.960 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-11T01:11:14.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:14.962 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-11T01:11:15.035 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-11T01:11:15.035 INFO:teuthology.orchestra.run.smithi039.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-11T01:11:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-11T01:11:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 1/35 2024-09-11T01:11:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2/35 2024-09-11T01:11:16.177 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 3/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 4/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 5/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_6 6/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 7/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9. 8/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x8 9/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 10/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 11/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 12/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 13/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-11T01:11:16.178 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 24/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-11T01:11:16.179 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-11T01:11:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-11T01:11:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-11T01:11:16.180 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 32/35 2024-09-11T01:11:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-11T01:11:16.181 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 34/35 2024-09-11T01:11:17.098 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout:Upgraded: 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.099 INFO:teuthology.orchestra.run.smithi039.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-11T01:11:17.100 INFO:teuthology.orchestra.run.smithi039.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-11T01:11:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-11T01:11:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-11T01:11:17.101 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: re2-1:20211101-20.el9.x86_64 2024-09-11T01:11:17.102 INFO:teuthology.orchestra.run.smithi039.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-11T01:11:17.103 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:17.103 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:17.446 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-test 2024-09-11T01:11:18.036 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:30 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:18.181 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:18.181 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:18.181 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout:Install 4 Packages 2024-09-11T01:11:18.182 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:18.183 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 51 M 2024-09-11T01:11:18.183 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 217 M 2024-09-11T01:11:18.183 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:11:18.423 INFO:teuthology.orchestra.run.smithi039.stdout:(1/4): libcephsqlite-19.3.0-4854.g8f219c47.el9. 744 kB/s | 162 kB 00:00 2024-09-11T01:11:18.982 INFO:teuthology.orchestra.run.smithi039.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-11T01:11:19.057 INFO:teuthology.orchestra.run.smithi039.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 102 kB/s | 64 kB 00:00 2024-09-11T01:11:19.765 INFO:teuthology.orchestra.run.smithi039.stdout:(4/4): ceph-test-19.3.0-4854.g8f219c47.el9.x86_ 32 MB/s | 50 MB 00:01 2024-09-11T01:11:19.768 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:11:19.768 INFO:teuthology.orchestra.run.smithi039.stdout:Total 32 MB/s | 51 MB 00:01 2024-09-11T01:11:19.796 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:11:19.819 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:11:19.819 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:11:20.237 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:11:20.237 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:11:20.790 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:11:20.881 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-11T01:11:20.924 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-11T01:11:20.959 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 3/4 2024-09-11T01:11:25.998 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 3/4 2024-09-11T01:11:26.085 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 4/4 2024-09-11T01:11:28.471 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 4/4 2024-09-11T01:11:28.471 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 1/4 2024-09-11T01:11:28.471 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2/4 2024-09-11T01:11:28.471 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-11T01:11:28.804 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:28.805 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:29.222 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph 2024-09-11T01:11:29.814 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:41 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:29.954 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:29.957 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:11:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:29.957 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-11T01:11:29.958 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-11T01:11:29.959 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-11T01:11:29.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:11:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:Install 37 Packages 2024-09-11T01:11:29.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 31 M 2024-09-11T01:11:29.961 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 109 M 2024-09-11T01:11:29.962 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:11:30.641 INFO:teuthology.orchestra.run.smithi039.stdout:(1/37): ceph-19.3.0-4854.g8f219c47.el9.x86_64.r 48 kB/s | 6.6 kB 00:00 2024-09-11T01:11:30.825 INFO:teuthology.orchestra.run.smithi039.stdout:(2/37): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-11T01:11:30.884 INFO:teuthology.orchestra.run.smithi039.stdout:(3/37): ceph-mds-19.3.0-4854.g8f219c47.el9.x86_ 6.3 MB/s | 2.4 MB 00:00 2024-09-11T01:11:30.984 INFO:teuthology.orchestra.run.smithi039.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4854.g8f21 2.8 MB/s | 291 kB 00:00 2024-09-11T01:11:31.067 INFO:teuthology.orchestra.run.smithi039.stdout:(5/37): ceph-mon-19.3.0-4854.g8f219c47.el9.x86_ 11 MB/s | 4.8 MB 00:00 2024-09-11T01:11:31.201 INFO:teuthology.orchestra.run.smithi039.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-11T01:11:31.243 INFO:teuthology.orchestra.run.smithi039.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 7.0 MB/s | 1.2 MB 00:00 2024-09-11T01:11:31.285 INFO:teuthology.orchestra.run.smithi039.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-11T01:11:31.469 INFO:teuthology.orchestra.run.smithi039.stdout:(9/37): ceph-osd-19.3.0-4854.g8f219c47.el9.x86_ 27 MB/s | 17 MB 00:00 2024-09-11T01:11:31.494 INFO:teuthology.orchestra.run.smithi039.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 598 kB/s | 125 kB 00:00 2024-09-11T01:11:31.527 INFO:teuthology.orchestra.run.smithi039.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 476 kB/s | 135 kB 00:00 2024-09-11T01:11:31.603 INFO:teuthology.orchestra.run.smithi039.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-09-11T01:11:31.628 INFO:teuthology.orchestra.run.smithi039.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-09-11T01:11:31.662 INFO:teuthology.orchestra.run.smithi039.stdout:(14/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.1 MB/s | 215 kB 00:00 2024-09-11T01:11:31.687 INFO:teuthology.orchestra.run.smithi039.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 413 kB/s | 35 kB 00:00 2024-09-11T01:11:31.712 INFO:teuthology.orchestra.run.smithi039.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 919 kB/s | 77 kB 00:00 2024-09-11T01:11:31.754 INFO:teuthology.orchestra.run.smithi039.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 552 kB/s | 51 kB 00:00 2024-09-11T01:11:31.780 INFO:teuthology.orchestra.run.smithi039.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 451 kB/s | 42 kB 00:00 2024-09-11T01:11:31.821 INFO:teuthology.orchestra.run.smithi039.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-11T01:11:31.855 INFO:teuthology.orchestra.run.smithi039.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-11T01:11:31.880 INFO:teuthology.orchestra.run.smithi039.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-11T01:11:31.906 INFO:teuthology.orchestra.run.smithi039.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-11T01:11:31.931 INFO:teuthology.orchestra.run.smithi039.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 705 kB/s | 18 kB 00:00 2024-09-11T01:11:31.956 INFO:teuthology.orchestra.run.smithi039.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-11T01:11:31.981 INFO:teuthology.orchestra.run.smithi039.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-09-11T01:11:32.007 INFO:teuthology.orchestra.run.smithi039.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-11T01:11:32.032 INFO:teuthology.orchestra.run.smithi039.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-11T01:11:32.057 INFO:teuthology.orchestra.run.smithi039.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-11T01:11:32.083 INFO:teuthology.orchestra.run.smithi039.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-11T01:11:32.108 INFO:teuthology.orchestra.run.smithi039.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-11T01:11:32.133 INFO:teuthology.orchestra.run.smithi039.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-11T01:11:32.275 INFO:teuthology.orchestra.run.smithi039.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 253 kB/s | 36 kB 00:00 2024-09-11T01:11:32.300 INFO:teuthology.orchestra.run.smithi039.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 468 kB/s | 90 kB 00:00 2024-09-11T01:11:32.326 INFO:teuthology.orchestra.run.smithi039.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-09-11T01:11:32.343 INFO:teuthology.orchestra.run.smithi039.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 1.2 MB/s | 20 kB 00:00 2024-09-11T01:11:32.368 INFO:teuthology.orchestra.run.smithi039.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-11T01:11:32.393 INFO:teuthology.orchestra.run.smithi039.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.5 MB/s | 427 kB 00:00 2024-09-11T01:11:32.397 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:11:32.397 INFO:teuthology.orchestra.run.smithi039.stdout:Total 13 MB/s | 31 MB 00:02 2024-09-11T01:11:32.552 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:11:32.595 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:11:32.595 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:11:33.075 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:11:33.076 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:11:34.500 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:11:34.554 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-11T01:11:34.587 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-11T01:11:34.623 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-11T01:11:34.697 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-11T01:11:34.868 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-11T01:11:34.901 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-11T01:11:34.995 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-11T01:11:35.089 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-11T01:11:35.137 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-11T01:11:35.171 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-11T01:11:35.201 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-11T01:11:35.236 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-11T01:11:35.269 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-11T01:11:35.310 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-11T01:11:35.353 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-11T01:11:35.387 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-11T01:11:35.437 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-11T01:11:35.485 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-11T01:11:35.560 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-11T01:11:35.711 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-11T01:11:35.795 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-11T01:11:35.847 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-11T01:11:35.885 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-11T01:11:35.951 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-11T01:11:36.023 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-11T01:11:36.216 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-11T01:11:36.280 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-11T01:11:36.360 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-11T01:11:36.458 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-11T01:11:36.518 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-11T01:11:36.655 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-11T01:11:36.827 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9. 32/37 2024-09-11T01:11:36.857 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 33/37 2024-09-11T01:11:36.885 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 33/37 2024-09-11T01:11:36.885 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:11:36.885 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-11T01:11:36.885 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T01:11:36.885 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-11T01:11:36.885 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:38.424 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 34/37 2024-09-11T01:11:38.453 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 34/37 2024-09-11T01:11:38.453 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:11:38.453 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-11T01:11:38.453 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T01:11:38.454 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-11T01:11:38.454 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:38.925 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 35/37 2024-09-11T01:11:38.950 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 35/37 2024-09-11T01:11:38.951 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:11:38.951 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-11T01:11:38.951 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T01:11:38.951 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-11T01:11:38.951 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:39.192 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 36/37 2024-09-11T01:11:39.218 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 36/37 2024-09-11T01:11:39.218 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:11:39.218 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-11T01:11:39.218 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T01:11:39.218 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-11T01:11:39.218 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:39.284 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 37/37 2024-09-11T01:11:40.911 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 37/37 2024-09-11T01:11:40.911 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/37 2024-09-11T01:11:40.911 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2/37 2024-09-11T01:11:40.911 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 3/37 2024-09-11T01:11:40.911 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 4/37 2024-09-11T01:11:40.911 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 5/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9. 6/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-11T01:11:40.912 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-11T01:11:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-11T01:11:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-11T01:11:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-11T01:11:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-11T01:11:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-11T01:11:40.913 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-11T01:11:40.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-11T01:11:40.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-11T01:11:40.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-11T01:11:40.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-11T01:11:40.914 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-11T01:11:40.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-11T01:11:40.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-11T01:11:40.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-11T01:11:40.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-11T01:11:40.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-11T01:11:40.915 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-11T01:11:41.873 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-11T01:11:41.873 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:41.873 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:11:41.873 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: lua-5.4.4-4.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-11T01:11:41.874 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ply-3.11-14.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-11T01:11:41.875 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:41.876 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:42.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-base 2024-09-11T01:11:42.847 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:54 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:42.942 INFO:teuthology.orchestra.run.smithi039.stdout:Package ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:11:42.986 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:42.987 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:11:42.987 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:43.051 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install cephadm 2024-09-11T01:11:43.634 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:55 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:43.771 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:43.771 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 783 k 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 790 k 2024-09-11T01:11:43.772 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:11:44.042 INFO:teuthology.orchestra.run.smithi039.stdout:cephadm-19.3.0-4854.g8f219c47.el9.noarch.rpm 2.8 MB/s | 783 kB 00:00 2024-09-11T01:11:44.043 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:11:44.043 INFO:teuthology.orchestra.run.smithi039.stdout:Total 2.8 MB/s | 783 kB 00:00 2024-09-11T01:11:44.043 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:11:44.049 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:11:44.049 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:11:44.055 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:11:44.055 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:11:44.131 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:11:44.597 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:11:44.698 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:11:45.078 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:11:45.342 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:11:45.342 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:45.342 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:11:45.343 INFO:teuthology.orchestra.run.smithi039.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:11:45.343 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:45.343 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:45.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-immutable-object-cache 2024-09-11T01:11:46.125 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:00:58 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:46.266 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:46.266 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 145 k 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 432 k 2024-09-11T01:11:46.267 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:11:46.475 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-immutable-object-cache-19.3.0-4854.g8f219c 699 kB/s | 145 kB 00:00 2024-09-11T01:11:46.475 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:11:46.475 INFO:teuthology.orchestra.run.smithi039.stdout:Total 695 kB/s | 145 kB 00:00 2024-09-11T01:11:46.476 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:11:46.484 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:11:46.484 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:11:46.545 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:11:46.545 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:11:46.694 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:11:46.750 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47. 1/1 2024-09-11T01:11:46.777 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47. 1/1 2024-09-11T01:11:46.777 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:11:46.777 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-11T01:11:46.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:47.727 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47. 1/1 2024-09-11T01:11:47.727 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:47.727 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:11:47.727 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:11:47.727 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:47.727 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:47.978 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr 2024-09-11T01:11:48.570 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:00 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:48.663 INFO:teuthology.orchestra.run.smithi039.stdout:Package ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:11:48.708 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:48.709 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:11:48.709 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:48.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-dashboard 2024-09-11T01:11:49.358 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:01 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:11:49.500 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-11T01:11:49.501 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout:Install 12 Packages 2024-09-11T01:11:49.502 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:49.503 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 19 M 2024-09-11T01:11:49.503 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 159 M 2024-09-11T01:11:49.503 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:11:49.953 INFO:teuthology.orchestra.run.smithi039.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4854.g8f2 100 kB/s | 16 kB 00:00 2024-09-11T01:11:49.978 INFO:teuthology.orchestra.run.smithi039.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4854.g8f 181 kB/s | 34 kB 00:00 2024-09-11T01:11:50.437 INFO:teuthology.orchestra.run.smithi039.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4854.g8f219c4 22 MB/s | 14 MB 00:00 2024-09-11T01:11:51.004 INFO:teuthology.orchestra.run.smithi039.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 263 kB/s | 269 kB 00:01 2024-09-11T01:11:51.054 INFO:teuthology.orchestra.run.smithi039.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 11 MB/s | 548 kB 00:00 2024-09-11T01:11:51.079 INFO:teuthology.orchestra.run.smithi039.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 778 kB/s | 19 kB 00:00 2024-09-11T01:11:51.146 INFO:teuthology.orchestra.run.smithi039.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 31 MB/s | 2.0 MB 00:00 2024-09-11T01:11:51.172 INFO:teuthology.orchestra.run.smithi039.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 5.7 MB/s | 144 kB 00:00 2024-09-11T01:11:51.197 INFO:teuthology.orchestra.run.smithi039.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-11T01:11:51.230 INFO:teuthology.orchestra.run.smithi039.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-09-11T01:11:51.522 INFO:teuthology.orchestra.run.smithi039.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 796 kB/s | 863 kB 00:01 2024-09-11T01:11:52.047 INFO:teuthology.orchestra.run.smithi039.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 493 kB/s | 1.0 MB 00:02 2024-09-11T01:11:52.052 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:11:52.053 INFO:teuthology.orchestra.run.smithi039.stdout:Total 7.6 MB/s | 19 MB 00:02 2024-09-11T01:11:52.107 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:11:52.130 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:11:52.130 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:11:52.429 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:11:52.429 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:11:53.048 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:11:53.141 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-11T01:11:53.184 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-11T01:11:53.231 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-11T01:11:53.260 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-11T01:11:53.367 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-11T01:11:53.502 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-11T01:11:53.594 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-11T01:11:53.658 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-11T01:11:53.686 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-11T01:11:53.719 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9 10/12 2024-09-11T01:11:57.097 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el 11/12 2024-09-11T01:11:57.155 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noa 12/12 2024-09-11T01:11:59.135 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noa 12/12 2024-09-11T01:11:59.135 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el 1/12 2024-09-11T01:11:59.135 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noa 2/12 2024-09-11T01:11:59.135 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9 3/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-11T01:11:59.136 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:11:59.477 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:11:59.478 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:11:59.745 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-11T01:12:00.338 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:12 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:00.480 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:00.481 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-11T01:12:00.481 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:12:00.481 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-11T01:12:00.481 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:00.481 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-11T01:12:00.481 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:00.482 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:00.483 INFO:teuthology.orchestra.run.smithi039.stdout:======================================================================================= 2024-09-11T01:12:00.483 INFO:teuthology.orchestra.run.smithi039.stdout:Install 12 Packages 2024-09-11T01:12:00.483 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:00.483 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 38 M 2024-09-11T01:12:00.483 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 212 M 2024-09-11T01:12:00.483 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:00.771 INFO:teuthology.orchestra.run.smithi039.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-11T01:12:00.830 INFO:teuthology.orchestra.run.smithi039.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 801 kB 00:00 2024-09-11T01:12:01.080 INFO:teuthology.orchestra.run.smithi039.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-48 16 MB/s | 7.4 MB 00:00 2024-09-11T01:12:01.105 INFO:teuthology.orchestra.run.smithi039.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 98 kB/s | 33 kB 00:00 2024-09-11T01:12:01.305 INFO:teuthology.orchestra.run.smithi039.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 192 kB/s | 38 kB 00:00 2024-09-11T01:12:01.364 INFO:teuthology.orchestra.run.smithi039.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-11T01:12:01.840 INFO:teuthology.orchestra.run.smithi039.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 515 kB/s | 245 kB 00:00 2024-09-11T01:12:02.040 INFO:teuthology.orchestra.run.smithi039.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-09-11T01:12:02.165 INFO:teuthology.orchestra.run.smithi039.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.0 MB/s | 250 kB 00:00 2024-09-11T01:12:03.049 INFO:teuthology.orchestra.run.smithi039.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.8 MB/s | 4.9 MB 00:01 2024-09-11T01:12:03.632 INFO:teuthology.orchestra.run.smithi039.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-09-11T01:12:17.015 INFO:teuthology.orchestra.run.smithi039.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 1.1 MB/s | 16 MB 00:14 2024-09-11T01:12:17.015 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:17.016 INFO:teuthology.orchestra.run.smithi039.stdout:Total 2.3 MB/s | 38 MB 00:16 2024-09-11T01:12:17.387 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:17.414 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:17.414 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:17.865 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:17.866 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:18.654 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:18.724 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-11T01:12:18.772 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-11T01:12:18.892 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-11T01:12:18.980 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-11T01:12:19.207 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-11T01:12:19.238 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-11T01:12:19.317 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-11T01:12:20.016 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-11T01:12:20.155 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-11T01:12:21.483 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T01:12:22.578 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-11T01:12:22.678 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219 12/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219 12/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219 1/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-11T01:12:25.235 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-11T01:12:25.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-11T01:12:25.236 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-11T01:12:25.732 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-11T01:12:25.733 INFO:teuthology.orchestra.run.smithi039.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-11T01:12:25.734 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-11T01:12:25.734 INFO:teuthology.orchestra.run.smithi039.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-11T01:12:25.734 INFO:teuthology.orchestra.run.smithi039.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-11T01:12:25.734 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:25.734 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:26.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-rook 2024-09-11T01:12:26.727 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:38 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:26.869 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:26.870 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:26.870 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:12:26.870 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:26.870 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-11T01:12:26.871 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout:Install 13 Packages 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:26.872 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 2.2 M 2024-09-11T01:12:26.873 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 26 M 2024-09-11T01:12:26.873 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:27.187 INFO:teuthology.orchestra.run.smithi039.stdout:(1/13): ceph-mgr-rook-19.3.0-4854.g8f219c47.el9 317 kB/s | 49 kB 00:00 2024-09-11T01:12:27.204 INFO:teuthology.orchestra.run.smithi039.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 111 kB/s | 19 kB 00:00 2024-09-11T01:12:27.221 INFO:teuthology.orchestra.run.smithi039.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 135 kB/s | 26 kB 00:00 2024-09-11T01:12:27.305 INFO:teuthology.orchestra.run.smithi039.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-09-11T01:12:27.346 INFO:teuthology.orchestra.run.smithi039.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-11T01:12:27.380 INFO:teuthology.orchestra.run.smithi039.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.7 MB/s | 279 kB 00:00 2024-09-11T01:12:27.405 INFO:teuthology.orchestra.run.smithi039.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 534 kB/s | 54 kB 00:00 2024-09-11T01:12:27.456 INFO:teuthology.orchestra.run.smithi039.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-09-11T01:12:27.481 INFO:teuthology.orchestra.run.smithi039.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-11T01:12:27.506 INFO:teuthology.orchestra.run.smithi039.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-11T01:12:27.532 INFO:teuthology.orchestra.run.smithi039.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-11T01:12:27.557 INFO:teuthology.orchestra.run.smithi039.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-11T01:12:27.590 INFO:teuthology.orchestra.run.smithi039.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-11T01:12:27.593 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:27.593 INFO:teuthology.orchestra.run.smithi039.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-11T01:12:27.662 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:27.679 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:27.680 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:27.777 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:27.778 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:28.303 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:28.472 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-11T01:12:28.536 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-11T01:12:28.607 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-11T01:12:28.651 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-11T01:12:28.693 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-11T01:12:28.800 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-11T01:12:28.910 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-11T01:12:28.963 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-11T01:12:29.728 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-11T01:12:29.791 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-11T01:12:29.825 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-11T01:12:29.883 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-11T01:12:29.912 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 13/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 13/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 1/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-11T01:12:30.679 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-11T01:12:30.680 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-11T01:12:30.680 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-11T01:12:30.680 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-11T01:12:31.031 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-11T01:12:31.032 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-11T01:12:31.032 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-11T01:12:31.032 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-11T01:12:31.032 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-11T01:12:31.032 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:31.032 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:31.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-mgr-cephadm 2024-09-11T01:12:31.882 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:43 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:32.021 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:32.022 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:32.022 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:12:32.022 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:32.022 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:32.022 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout:Install 6 Packages 2024-09-11T01:12:32.023 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:32.024 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 7.0 M 2024-09-11T01:12:32.024 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 33 M 2024-09-11T01:12:32.024 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:32.692 INFO:teuthology.orchestra.run.smithi039.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-11T01:12:32.717 INFO:teuthology.orchestra.run.smithi039.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4854.g8f219c47.e 667 kB/s | 154 kB 00:00 2024-09-11T01:12:32.776 INFO:teuthology.orchestra.run.smithi039.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-11T01:12:32.801 INFO:teuthology.orchestra.run.smithi039.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 691 kB/s | 58 kB 00:00 2024-09-11T01:12:32.826 INFO:teuthology.orchestra.run.smithi039.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-11T01:12:34.118 INFO:teuthology.orchestra.run.smithi039.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.7 MB/s | 6.0 MB 00:01 2024-09-11T01:12:34.121 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:34.122 INFO:teuthology.orchestra.run.smithi039.stdout:Total 3.3 MB/s | 7.0 MB 00:02 2024-09-11T01:12:34.198 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:34.213 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:34.213 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:34.318 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:34.318 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:34.634 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:34.801 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-11T01:12:34.855 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-11T01:12:35.417 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-11T01:12:35.518 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-11T01:12:35.601 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-11T01:12:35.632 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 6/6 2024-09-11T01:12:36.371 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 6/6 2024-09-11T01:12:36.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 1/6 2024-09-11T01:12:36.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-11T01:12:36.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-11T01:12:36.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-11T01:12:36.371 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-11T01:12:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-11T01:12:36.661 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:36.661 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:36.662 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:36.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-fuse 2024-09-11T01:12:37.499 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:49 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:37.640 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:37.640 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout:Installing dependencies: 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout:Install 2 Packages 2024-09-11T01:12:37.641 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:37.642 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 980 k 2024-09-11T01:12:37.642 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 2.9 M 2024-09-11T01:12:37.642 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:38.304 INFO:teuthology.orchestra.run.smithi039.stdout:(1/2): ceph-fuse-19.3.0-4854.g8f219c47.el9.x86_ 1.6 MB/s | 900 kB 00:00 2024-09-11T01:12:38.429 INFO:teuthology.orchestra.run.smithi039.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 119 kB/s | 80 kB 00:00 2024-09-11T01:12:38.430 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:38.430 INFO:teuthology.orchestra.run.smithi039.stdout:Total 1.2 MB/s | 980 kB 00:00 2024-09-11T01:12:38.444 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:38.470 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:38.470 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:38.560 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:38.560 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:38.767 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:38.945 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-11T01:12:39.014 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2/2 2024-09-11T01:12:39.722 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2/2 2024-09-11T01:12:39.722 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 1/2 2024-09-11T01:12:40.045 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-11T01:12:40.045 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:40.045 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:40.045 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-11T01:12:40.045 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:40.045 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:40.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install ceph-volume 2024-09-11T01:12:40.961 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:52 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:41.100 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:41.100 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repository Size 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 285 k 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 1.3 M 2024-09-11T01:12:41.101 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:41.309 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-volume-19.3.0-4854.g8f219c47.el9.noarch.rp 1.3 MB/s | 285 kB 00:00 2024-09-11T01:12:41.309 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:41.310 INFO:teuthology.orchestra.run.smithi039.stdout:Total 1.3 MB/s | 285 kB 00:00 2024-09-11T01:12:41.310 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:41.319 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:41.319 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:41.350 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:41.350 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:41.582 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:41.653 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:12:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:12:41.670 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:12:41.670 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-11T01:12:41.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:42.745 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 1/1 2024-09-11T01:12:42.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:42.745 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:42.745 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-11T01:12:42.745 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:42.745 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:43.047 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados-devel 2024-09-11T01:12:43.643 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:55 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 127 k 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 456 k 2024-09-11T01:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:43.997 INFO:teuthology.orchestra.run.smithi039.stdout:librados-devel-19.3.0-4854.g8f219c47.el9.x86_64 596 kB/s | 127 kB 00:00 2024-09-11T01:12:43.997 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:43.998 INFO:teuthology.orchestra.run.smithi039.stdout:Total 592 kB/s | 127 kB 00:00 2024-09-11T01:12:43.998 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:44.004 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:44.004 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:44.077 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:44.077 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:44.244 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:44.350 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:44.793 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:45.052 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:45.052 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:45.052 INFO:teuthology.orchestra.run.smithi039.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:12:45.052 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:45.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install libcephfs2 2024-09-11T01:12:45.899 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:57 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:45.994 INFO:teuthology.orchestra.run.smithi039.stdout:Package libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:46.042 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:46.043 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:46.043 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:46.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install libcephfs-devel 2024-09-11T01:12:46.701 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:01:58 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:46.841 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:46.842 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:46.842 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-09-11T01:12:46.842 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 32 k 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 142 k 2024-09-11T01:12:46.843 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:46.992 INFO:teuthology.orchestra.run.smithi039.stdout:libcephfs-devel-19.3.0-4854.g8f219c47.el9.x86_6 213 kB/s | 32 kB 00:00 2024-09-11T01:12:46.993 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:46.993 INFO:teuthology.orchestra.run.smithi039.stdout:Total 211 kB/s | 32 kB 00:00 2024-09-11T01:12:46.993 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:46.997 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:46.997 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:47.024 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:47.024 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:47.117 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:47.226 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:47.614 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:47.868 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:47.868 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:47.868 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:47.869 INFO:teuthology.orchestra.run.smithi039.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:12:47.869 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:47.869 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:48.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librados2 2024-09-11T01:12:48.661 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:00 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:48.754 INFO:teuthology.orchestra.run.smithi039.stdout:Package librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:48.800 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:48.801 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:48.801 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:48.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install librbd1 2024-09-11T01:12:49.457 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:01 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:49.550 INFO:teuthology.orchestra.run.smithi039.stdout:Package librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:49.596 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:49.597 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:49.597 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:49.664 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rados 2024-09-11T01:12:50.253 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:02 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:50.346 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:50.392 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:50.393 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:50.393 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:50.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rgw 2024-09-11T01:12:51.043 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:03 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:51.135 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:51.181 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:51.182 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:51.183 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:51.252 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-cephfs 2024-09-11T01:12:51.840 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:03 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:51.932 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:51.978 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:51.979 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:51.979 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:52.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install python3-rbd 2024-09-11T01:12:52.635 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:04 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:52.728 INFO:teuthology.orchestra.run.smithi039.stdout:Package python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 is already installed. 2024-09-11T01:12:52.775 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:52.776 INFO:teuthology.orchestra.run.smithi039.stdout:Nothing to do. 2024-09-11T01:12:52.776 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:52.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-fuse 2024-09-11T01:12:53.431 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:05 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:53.568 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:53.569 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-09-11T01:12:53.569 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:53.569 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:53.569 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 85 k 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 231 k 2024-09-11T01:12:53.570 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:53.971 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-fuse-19.3.0-4854.g8f219c47.el9.x86_64.rpm 213 kB/s | 85 kB 00:00 2024-09-11T01:12:53.971 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:53.971 INFO:teuthology.orchestra.run.smithi039.stdout:Total 212 kB/s | 85 kB 00:00 2024-09-11T01:12:53.972 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:53.979 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:53.979 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:54.050 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:54.050 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:54.214 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:54.324 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:54.692 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:54.970 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:54.970 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:54.970 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:54.970 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:12:54.970 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:54.970 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:55.225 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-mirror 2024-09-11T01:12:55.823 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:07 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:55.962 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:55.963 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:55.963 INFO:teuthology.orchestra.run.smithi039.stdout: Package Arch Version Repo Size 2024-09-11T01:12:55.963 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:55.963 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 3.2 M 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 14 M 2024-09-11T01:12:55.964 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:56.666 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-mirror-19.3.0-4854.g8f219c47.el9.x86_64.rpm 4.5 MB/s | 3.2 MB 00:00 2024-09-11T01:12:56.666 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:Total 4.5 MB/s | 3.2 MB 00:00 2024-09-11T01:12:56.667 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:56.678 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:56.678 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:56.769 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:56.769 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:57.235 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:12:57.291 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:57.318 INFO:teuthology.orchestra.run.smithi039.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-11T01:12:57.318 INFO:teuthology.orchestra.run.smithi039.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-11T01:12:57.318 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T01:12:57.318 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-11T01:12:57.318 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:58.543 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:12:58.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:58.544 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:12:58.544 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:12:58.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:58.544 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:12:58.782 DEBUG:teuthology.orchestra.run.smithi039:> sudo yum -y install rbd-nbd 2024-09-11T01:12:59.378 INFO:teuthology.orchestra.run.smithi039.stdout:Last metadata expiration check: 0:02:11 ago on Wed 11 Sep 2024 01:10:48 AM UTC. 2024-09-11T01:12:59.515 INFO:teuthology.orchestra.run.smithi039.stdout:Dependencies resolved. 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout: Package Architecture Version Repository Size 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout:Installing: 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction Summary 2024-09-11T01:12:59.516 INFO:teuthology.orchestra.run.smithi039.stdout:================================================================================ 2024-09-11T01:12:59.517 INFO:teuthology.orchestra.run.smithi039.stdout:Install 1 Package 2024-09-11T01:12:59.517 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:12:59.517 INFO:teuthology.orchestra.run.smithi039.stdout:Total download size: 171 k 2024-09-11T01:12:59.517 INFO:teuthology.orchestra.run.smithi039.stdout:Installed size: 490 k 2024-09-11T01:12:59.517 INFO:teuthology.orchestra.run.smithi039.stdout:Downloading Packages: 2024-09-11T01:12:59.727 INFO:teuthology.orchestra.run.smithi039.stdout:rbd-nbd-19.3.0-4854.g8f219c47.el9.x86_64.rpm 812 kB/s | 171 kB 00:00 2024-09-11T01:12:59.728 INFO:teuthology.orchestra.run.smithi039.stdout:-------------------------------------------------------------------------------- 2024-09-11T01:12:59.728 INFO:teuthology.orchestra.run.smithi039.stdout:Total 807 kB/s | 171 kB 00:00 2024-09-11T01:12:59.728 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction check 2024-09-11T01:12:59.737 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction check succeeded. 2024-09-11T01:12:59.737 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction test 2024-09-11T01:12:59.806 INFO:teuthology.orchestra.run.smithi039.stdout:Transaction test succeeded. 2024-09-11T01:12:59.806 INFO:teuthology.orchestra.run.smithi039.stdout:Running transaction 2024-09-11T01:12:59.967 INFO:teuthology.orchestra.run.smithi039.stdout: Preparing : 1/1 2024-09-11T01:13:00.079 INFO:teuthology.orchestra.run.smithi039.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:13:00.507 INFO:teuthology.orchestra.run.smithi039.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:13:00.767 INFO:teuthology.orchestra.run.smithi039.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 1/1 2024-09-11T01:13:00.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:00.767 INFO:teuthology.orchestra.run.smithi039.stdout:Installed: 2024-09-11T01:13:00.767 INFO:teuthology.orchestra.run.smithi039.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-11T01:13:00.767 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:00.767 INFO:teuthology.orchestra.run.smithi039.stdout:Complete! 2024-09-11T01:13:01.045 DEBUG:teuthology.parallel:result is None 2024-09-11T01:13:01.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T01:13:01.207 DEBUG:teuthology.orchestra.run.smithi039:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-11T01:13:01.229 INFO:teuthology.orchestra.run.smithi039.stdout:19.3.0-4854.g8f219c47.el9 2024-09-11T01:13:01.230 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-11T01:13:01.230 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-11T01:13:01.232 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T01:13:01.232 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:01.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T01:13:01.296 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T01:13:01.296 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:01.296 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T01:13:01.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T01:13:01.428 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T01:13:01.429 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:01.429 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T01:13:01.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T01:13:01.561 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T01:13:01.562 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:01.562 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T01:13:01.628 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T01:13:01.693 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T01:13:01.775 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T01:13:01.776 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2024-09-11T01:13:01.802 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T01:13:01.802 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T01:13:01.868 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T01:13:01.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T01:13:01.941 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T01:13:01.942 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-11T01:13:01.942 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-lz4} rados supported-random-distro$/{centos_latest}}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899880', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 9167, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'targets': {'smithi039.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDi3saszkDpcglMo1+hBVtJhZXO0hCIWGWZzsgbgqJg6rJ7DGWjLKVbqPXCUiRfLLHAjpjcIYEeet20R/yEUClM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'lost_unfound': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_17:16:14', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T01:13:01.942 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T01:13:01.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T01:13:02.056 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:02.056 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T01:13:02.113 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T01:13:02.113 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-11 01:12:57.580810012 +0000 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-11 01:10:08.293180109 +0000 2024-09-11T01:13:02.169 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-11 01:10:08.293180109 +0000 2024-09-11T01:13:02.170 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-11 01:10:08.293180109 +0000 2024-09-11T01:13:02.170 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T01:13:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-11T01:13:02.233 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-11T01:13:02.234 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000275285 s, 1.9 MB/s 2024-09-11T01:13:02.235 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T01:13:02.292 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2024-09-11T01:13:02.348 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T01:13:02.348 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:13:02.348 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-11T01:13:02.348 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:13:02.348 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T01:13:02.349 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-11 01:12:57.580810012 +0000 2024-09-11T01:13:02.349 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-11 01:10:08.638172989 +0000 2024-09-11T01:13:02.349 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-11 01:10:08.638172989 +0000 2024-09-11T01:13:02.349 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-11 01:10:08.638172989 +0000 2024-09-11T01:13:02.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T01:13:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-11T01:13:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-11T01:13:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000290466 s, 1.8 MB/s 2024-09-11T01:13:02.414 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T01:13:02.471 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-11 01:12:57.580810012 +0000 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-11 01:10:08.972166096 +0000 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-11 01:10:08.972166096 +0000 2024-09-11T01:13:02.527 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-11 01:10:08.972166096 +0000 2024-09-11T01:13:02.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T01:13:02.591 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-11T01:13:02.591 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-11T01:13:02.591 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000265278 s, 1.9 MB/s 2024-09-11T01:13:02.592 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T01:13:02.648 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2024-09-11T01:13:02.704 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T01:13:02.704 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:13:02.704 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-11T01:13:02.704 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:13:02.704 INFO:teuthology.orchestra.run.smithi039.stdout:Context: system_u:object_r:device_t:s0 2024-09-11T01:13:02.704 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2024-09-11 01:12:57.581809991 +0000 2024-09-11T01:13:02.705 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2024-09-11 01:10:09.304159245 +0000 2024-09-11T01:13:02.705 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2024-09-11 01:10:09.304159245 +0000 2024-09-11T01:13:02.705 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2024-09-11 01:10:09.304159245 +0000 2024-09-11T01:13:02.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T01:13:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2024-09-11T01:13:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2024-09-11T01:13:02.768 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000319787 s, 1.6 MB/s 2024-09-11T01:13:02.770 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T01:13:02.828 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:13:02.828 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-11T01:13:02.828 INFO:tasks.ceph:Generating config... 2024-09-11T01:13:02.829 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-11T01:13:02.830 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-11T01:13:02.830 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-09-11T01:13:02.830 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-09-11T01:13:02.830 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-11T01:13:02.830 INFO:tasks.ceph:[global] ms type = async 2024-09-11T01:13:02.830 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T01:13:02.830 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T01:13:02.830 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T01:13:02.830 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T01:13:02.830 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T01:13:02.831 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T01:13:02.831 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-11T01:13:02.832 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-11T01:13:02.832 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T01:13:02.832 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T01:13:02.920 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T01:13:02.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T01:13:02.966 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T01:13:03.032 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.39'), ('mon.b', '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]'), ('mon.c', '[v2:172.21.15.39:3302,v1:172.21.15.39:6791]')] 2024-09-11T01:13:03.032 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],[v2:172.21.15.39:3302,v1:172.21.15.39:6791]', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-11T01:13:03.033 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:03.033 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T01:13:03.088 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.39 --addv b '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]' --addv c '[v2:172.21.15.39:3302,v1:172.21.15.39:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: generated fsid bcb624ec-d02e-4f87-a117-bb3a3b752391 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:setting min_mon_release = quincy 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 0 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:fsid bcb624ec-d02e-4f87-a117-bb3a3b752391 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:last_changed 2024-09-11T01:13:03.176883+0000 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-11T01:13:03.176883+0000 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:min_mon_release 17 (quincy) 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:election_strategy: 1 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:0: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.a 2024-09-11T01:13:03.178 INFO:teuthology.orchestra.run.smithi039.stdout:1: [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] mon.b 2024-09-11T01:13:03.179 INFO:teuthology.orchestra.run.smithi039.stdout:2: [v2:172.21.15.39:3302/0,v1:172.21.15.39:6791/0] mon.c 2024-09-11T01:13:03.179 INFO:teuthology.orchestra.run.smithi039.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T01:13:03.180 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T01:13:03.195 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID bcb624ec-d02e-4f87-a117-bb3a3b752391... 2024-09-11T01:13:03.196 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.280 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2024-09-11T01:13:03.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.282 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790],[v2:172.21.15.39:3302,v1:172.21.15.39:6791]" 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: mon client directed command retry = 5 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: mon election default strategy = 1 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: ms bind msgr1 = False 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: ms bind msgr2 = True 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: ms inject socket failures = 5000 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: ms type = async 2024-09-11T01:13:03.283 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = bcb624ec-d02e-4f87-a117-bb3a3b752391 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.284 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug trim objects = true 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2024-09-11T01:13:03.285 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.286 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2024-09-11T01:13:03.287 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2024-09-11T01:13:03.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.287 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2024-09-11T01:13:03.287 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2024-09-11T01:13:03.287 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2024-09-11T01:13:03.287 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore compression algorithm = lz4 2024-09-11T01:13:03.288 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore compression mode = aggressive 2024-09-11T01:13:03.288 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2024-09-11T01:13:03.288 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore write v2 random = True 2024-09-11T01:13:03.288 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore zero block detection = True 2024-09-11T01:13:03.288 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 20 2024-09-11T01:13:03.288 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 20 2024-09-11T01:13:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-11T01:13:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2024-09-11T01:13:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 10 2024-09-11T01:13:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T01:13:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2024-09-11T01:13:03.289 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2024-09-11T01:13:03.290 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify cached snaps = True 2024-09-11T01:13:03.290 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify missing on start = True 2024-09-11T01:13:03.290 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2024-09-11T01:13:03.291 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock override recovery settings = True 2024-09-11T01:13:03.291 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2024-09-11T01:13:03.291 INFO:teuthology.orchestra.run.smithi039.stdout: osd op queue = debug_random 2024-09-11T01:13:03.291 INFO:teuthology.orchestra.run.smithi039.stdout: osd op queue cut off = debug_random 2024-09-11T01:13:03.292 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.292 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2024-09-11T01:13:03.292 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-11T01:13:03.292 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2024-09-11T01:13:03.292 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-11T01:13:03.293 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-11T01:13:03.293 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-11T01:13:03.293 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-11T01:13:03.293 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2024-09-11T01:13:03.293 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.294 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2024-09-11T01:13:03.294 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2024-09-11T01:13:03.294 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2024-09-11T01:13:03.294 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2024-09-11T01:13:03.294 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2024-09-11T01:13:03.294 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2024-09-11T01:13:03.295 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2024-09-11T01:13:03.295 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2024-09-11T01:13:03.295 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2024-09-11T01:13:03.295 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2024-09-11T01:13:03.296 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2024-09-11T01:13:03.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.296 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T01:13:03.296 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T01:13:03.296 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T01:13:03.296 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.297 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T01:13:03.297 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T01:13:03.297 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T01:13:03.297 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.297 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2024-09-11T01:13:03.297 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2024-09-11T01:13:03.298 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.298 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2024-09-11T01:13:03.298 INFO:teuthology.orchestra.run.smithi039.stdout: mon scrub interval = 300 2024-09-11T01:13:03.298 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:03.298 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2024-09-11T01:13:03.298 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2024-09-11T01:13:03.299 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2024-09-11T01:13:03.299 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2024-09-11T01:13:03.299 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T01:13:03.299 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T01:13:03.299 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2024-09-11T01:13:03.299 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2024-09-11T01:13:03.300 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2024-09-11T01:13:03.301 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T01:13:03.301 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T01:13:03.359 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T01:13:03.360 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:03.360 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T01:13:03.376 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:03.376 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T01:13:03.432 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:13:03.432 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:03.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T01:13:03.433 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T01:13:03.507 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:03.508 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:03.564 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T01:13:03.564 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T01:13:03.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T01:13:03.657 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T01:13:03.659 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T01:13:03.660 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T01:13:03.660 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T01:13:03.660 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-11T01:13:03.660 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T01:13:03.685 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:13:03.686 INFO:tasks.ceph:role: osd.0 2024-09-11T01:13:03.686 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:13:03.686 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T01:13:03.779 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T01:13:03.780 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T01:13:05.128 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-11T01:13:05.131 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-11T01:13:05.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T01:13:05.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T01:13:05.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T01:13:05.315 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:13:05.315 INFO:tasks.ceph:role: osd.1 2024-09-11T01:13:05.315 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:13:05.316 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T01:13:05.411 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T01:13:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T01:13:05.411 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T01:13:05.412 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T01:13:06.733 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-11T01:13:06.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-11T01:13:06.736 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T01:13:06.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T01:13:06.846 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T01:13:06.914 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:13:06.914 INFO:tasks.ceph:role: osd.2 2024-09-11T01:13:06.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:13:06.914 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T01:13:07.009 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T01:13:07.010 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T01:13:08.358 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2024-09-11T01:13:08.360 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2024-09-11T01:13:08.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T01:13:08.398 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T01:13:08.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:08.582 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:08.580+0000 7f4a6902c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T01:13:08.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:08.580+0000 7f4a6902c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T01:13:08.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:08.580+0000 7f4a6902c640 -1 bdev(0x556b6b15b800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T01:13:08.583 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:08.580+0000 7f4a6902c640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T01:13:11.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T01:13:11.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:11.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:11.881+0000 7fda69032640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T01:13:11.884 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:11.882+0000 7fda69032640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T01:13:11.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:11.882+0000 7fda69032640 -1 bdev(0x563f04563800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T01:13:11.885 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:11.882+0000 7fda69032640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T01:13:14.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T01:13:14.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:14.940+0000 7fe1c7215640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T01:13:14.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:14.940+0000 7fe1c7215640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T01:13:14.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:14.940+0000 7fe1c7215640 -1 bdev(0x5572e704f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T01:13:14.943 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:13:14.940+0000 7fe1c7215640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T01:13:18.134 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T01:13:18.164 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T01:13:18.164 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:18.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T01:13:18.234 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:18.234 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T01:13:18.303 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:18.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T01:13:18.371 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:18.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T01:13:18.438 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T01:13:18.438 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCf7uBm/3odJxAAynSusduhfmq0eIj+k53MtA== 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCk7uBm+PCqIhAAvj8tGISfL+3+My33U9KpJQ== 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCn7uBmPRWnNBAANDCOEAIFZUurjaqLsKYU1A== 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2024-09-11T01:13:18.504 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQCq7uBmP/UiOBAA6mWX7B0jUwgCKlSjN4wAuQ== 2024-09-11T01:13:18.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T01:13:18.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T01:13:18.653 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T01:13:18.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T01:13:18.758 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T01:13:18.758 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T01:13:18.783 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T01:13:18.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T01:13:18.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T01:13:18.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T01:13:19.103 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T01:13:19.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T01:13:19.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T01:13:19.312 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T01:13:19.338 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:13:19.394 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T01:13:19.394 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T01:13:19.395 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T01:13:19.437 INFO:tasks.ceph.mon.a:Started 2024-09-11T01:13:19.437 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T01:13:19.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T01:13:19.439 INFO:tasks.ceph.mon.b:Started 2024-09-11T01:13:19.439 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T01:13:19.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T01:13:19.442 INFO:tasks.ceph.mon.c:Started 2024-09-11T01:13:19.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph config set mgr mgr_pool false --force 2024-09-11T01:13:19.839 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T01:13:19.840 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T01:13:19.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T01:13:19.842 INFO:tasks.ceph.mgr.x:Started 2024-09-11T01:13:19.842 DEBUG:tasks.ceph:set 0 configs 2024-09-11T01:13:19.842 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T01:13:19.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T01:13:20.045 INFO:teuthology.orchestra.run.smithi039.stderr:adjusted tunables profile to default 2024-09-11T01:13:20.057 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T01:13:20.057 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T01:13:20.057 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:20.057 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T01:13:20.083 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:20.083 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T01:13:20.149 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2024-09-11T01:13:20.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T01:13:20.215 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new f5f0de43-07f1-4db0-9ad7-4564c8fd2953 0 2024-09-11T01:13:20.448 INFO:teuthology.orchestra.run.smithi039.stdout:0 2024-09-11T01:13:20.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 5df6a606-9f1b-4a88-8879-e1661e7956c2 1 2024-09-11T01:13:20.655 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-11T01:13:20.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new ee7baf02-576b-4245-9572-bfd1ab1e3a0d 2 2024-09-11T01:13:20.883 INFO:teuthology.orchestra.run.smithi039.stdout:2 2024-09-11T01:13:20.894 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T01:13:20.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T01:13:20.896 INFO:tasks.ceph.osd.0:Started 2024-09-11T01:13:20.896 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T01:13:20.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T01:13:20.899 INFO:tasks.ceph.osd.1:Started 2024-09-11T01:13:20.899 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T01:13:20.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T01:13:20.901 INFO:tasks.ceph.osd.2:Started 2024-09-11T01:13:20.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:21.181 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:21.181 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":5,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:20.872658+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:21.192 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T01:13:21.192 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T01:13:21.494 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:21.749 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-11T01:13:21.749 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":5,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:20.872658+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:21.759 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-11T01:13:22.344 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:13:22.342+0000 7f5000a1c640 -1 Falling back to public interface 2024-09-11T01:13:22.353 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:13:22.352+0000 7f15f28e4640 -1 Falling back to public interface 2024-09-11T01:13:22.600 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:13:22.598+0000 7f9a185fd640 -1 Falling back to public interface 2024-09-11T01:13:25.405 INFO:tasks.ceph.mgr.x.smithi039.stderr:2024-09-11T01:13:25.403+0000 7f3ebe57d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T01:13:25.406 INFO:tasks.ceph.mgr.x.smithi039.stderr:2024-09-11T01:13:25.405+0000 7f3ebe57d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T01:13:26.134 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:13:26.133+0000 7f9a185fd640 -1 osd.2 0 log_to_monitors true 2024-09-11T01:13:26.169 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:13:26.168+0000 7f5000a1c640 -1 osd.0 0 log_to_monitors true 2024-09-11T01:13:26.447 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:13:26.446+0000 7f15f28e4640 -1 osd.1 0 log_to_monitors true 2024-09-11T01:13:27.419 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:13:27.417+0000 7f9a1458c640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T01:13:27.427 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:13:27.426+0000 7f9a103b3640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:13:27.976 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:13:27.974+0000 7f4ffc9ab640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T01:13:27.978 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:13:27.977+0000 7f4ff47ca640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:13:28.062 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:28.353 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-11T01:13:28.354 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":7,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:27.401839+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:28.365 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-11T01:13:28.425 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:13:28.423+0000 7f15ee873640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T01:13:28.427 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:13:28.426+0000 7f15ea69a640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:13:34.668 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:34.970 INFO:teuthology.misc.health.smithi039.stdout: 2024-09-11T01:13:34.970 INFO:teuthology.misc.health.smithi039.stdout:{"epoch":10,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:30.426421+0000","last_up_change":"2024-09-11T01:13:29.413330+0000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":7462012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":7462012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":7462012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":7462012}]},"public_addr":"172.21.15.39:6800/7462012","cluster_addr":"172.21.15.39:6801/7462012","heartbeat_back_addr":"172.21.15.39:6803/7462012","heartbeat_front_addr":"172.21.15.39:6802/7462012","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3162293523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":3162293523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":3162293523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3162293523}]},"public_addr":"172.21.15.39:6804/3162293523","cluster_addr":"172.21.15.39:6805/3162293523","heartbeat_back_addr":"172.21.15.39:6807/3162293523","heartbeat_front_addr":"172.21.15.39:6806/3162293523","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":3298933028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":3298933028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":3298933028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3298933028}]},"public_addr":"172.21.15.39:6808/3298933028","cluster_addr":"172.21.15.39:6809/3298933028","heartbeat_back_addr":"172.21.15.39:6811/3298933028","heartbeat_front_addr":"172.21.15.39:6810/3298933028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:34.981 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-11T01:13:34.981 INFO:tasks.ceph:Creating RBD pool 2024-09-11T01:13:34.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-11T01:13:35.472 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'rbd' created 2024-09-11T01:13:35.488 DEBUG:teuthology.orchestra.run.smithi039:> rbd --cluster ceph pool init rbd 2024-09-11T01:13:38.542 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T01:13:38.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-11T01:13:38.544 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T01:13:38.854 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:38.868 INFO:teuthology.orchestra.run.smithi039.stdout:[{"version":3,"timestamp":"2024-09-11T01:13:27.975711+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"9552.639146"}]}] 2024-09-11T01:13:38.868 INFO:tasks.ceph_manager:config epoch is 3 2024-09-11T01:13:38.868 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T01:13:38.868 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T01:13:38.869 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T01:13:39.217 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:39.229 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":4,"flags":0,"active_gid":4110,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":1342718687}]},"active_addr":"172.21.15.39:6812/1342718687","active_change":"2024-09-11T01:13:24.387762+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":4259426363}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":4112911599}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.39:0","nonce":3016637982}]}]} 2024-09-11T01:13:39.231 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T01:13:39.231 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T01:13:39.232 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:39.533 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:39.533 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:38.508203+0000","last_up_change":"2024-09-11T01:13:29.413330+0000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":7462012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":7462012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":7462012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":7462012}]},"public_addr":"172.21.15.39:6800/7462012","cluster_addr":"172.21.15.39:6801/7462012","heartbeat_back_addr":"172.21.15.39:6803/7462012","heartbeat_front_addr":"172.21.15.39:6802/7462012","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3162293523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":3162293523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":3162293523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3162293523}]},"public_addr":"172.21.15.39:6804/3162293523","cluster_addr":"172.21.15.39:6805/3162293523","heartbeat_back_addr":"172.21.15.39:6807/3162293523","heartbeat_front_addr":"172.21.15.39:6806/3162293523","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":3298933028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":3298933028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":3298933028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3298933028}]},"public_addr":"172.21.15.39:6808/3298933028","cluster_addr":"172.21.15.39:6809/3298933028","heartbeat_back_addr":"172.21.15.39:6811/3298933028","heartbeat_front_addr":"172.21.15.39:6810/3298933028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:39.545 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T01:13:39.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:39.847 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:39.847 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:38.508203+0000","last_up_change":"2024-09-11T01:13:29.413330+0000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":7462012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":7462012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":7462012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":7462012}]},"public_addr":"172.21.15.39:6800/7462012","cluster_addr":"172.21.15.39:6801/7462012","heartbeat_back_addr":"172.21.15.39:6803/7462012","heartbeat_front_addr":"172.21.15.39:6802/7462012","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3162293523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":3162293523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":3162293523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3162293523}]},"public_addr":"172.21.15.39:6804/3162293523","cluster_addr":"172.21.15.39:6805/3162293523","heartbeat_back_addr":"172.21.15.39:6807/3162293523","heartbeat_front_addr":"172.21.15.39:6806/3162293523","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":3298933028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":3298933028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":3298933028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3298933028}]},"public_addr":"172.21.15.39:6808/3298933028","cluster_addr":"172.21.15.39:6809/3298933028","heartbeat_back_addr":"172.21.15.39:6811/3298933028","heartbeat_front_addr":"172.21.15.39:6810/3298933028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:39.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:13:39.860 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:13:39.861 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:13:40.003 INFO:teuthology.orchestra.run.smithi039.stdout:34359738374 2024-09-11T01:13:40.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:13:40.004 INFO:teuthology.orchestra.run.smithi039.stdout:34359738373 2024-09-11T01:13:40.004 INFO:teuthology.orchestra.run.smithi039.stdout:38654705669 2024-09-11T01:13:40.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:13:40.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:13:40.353 INFO:teuthology.orchestra.run.smithi039.stdout:38654705667 2024-09-11T01:13:40.354 INFO:teuthology.orchestra.run.smithi039.stdout:34359738372 2024-09-11T01:13:40.357 INFO:teuthology.orchestra.run.smithi039.stdout:34359738371 2024-09-11T01:13:40.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.0 2024-09-11T01:13:40.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705667 for osd.1 2024-09-11T01:13:40.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738372 for osd.2 2024-09-11T01:13:41.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:13:41.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:13:41.374 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:13:41.698 INFO:teuthology.orchestra.run.smithi039.stdout:38654705669 2024-09-11T01:13:41.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2024-09-11T01:13:41.713 DEBUG:teuthology.parallel:result is None 2024-09-11T01:13:41.714 INFO:teuthology.orchestra.run.smithi039.stdout:34359738374 2024-09-11T01:13:41.720 INFO:teuthology.orchestra.run.smithi039.stdout:34359738373 2024-09-11T01:13:41.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.2 2024-09-11T01:13:41.728 DEBUG:teuthology.parallel:result is None 2024-09-11T01:13:41.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.0 2024-09-11T01:13:41.733 DEBUG:teuthology.parallel:result is None 2024-09-11T01:13:41.734 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T01:13:41.734 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:42.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:42.067 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:42.079 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-11T01:13:40.398095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85616,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029904,"statfs":{"total":289910292480,"available":289822621696,"internally_reserved":0,"allocated":376832,"data_stored":87749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":87223632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533864+0000","last_change":"2024-09-11T01:13:38.533864+0000","last_active":"2024-09-11T01:13:38.533864+0000","last_peered":"2024-09-11T01:13:38.533864+0000","last_clean":"2024-09-11T01:13:38.533864+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:38.533864+0000","last_undegraded":"2024-09-11T01:13:38.533864+0000","last_fullsized":"2024-09-11T01:13:38.533864+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:21.523374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.534017+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:38.534017+0000","last_peered":"2024-09-11T01:13:38.534017+0000","last_clean":"2024-09-11T01:13:38.534017+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:38.534017+0000","last_undegraded":"2024-09-11T01:13:38.534017+0000","last_fullsized":"2024-09-11T01:13:38.534017+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502309+0000","last_became_peered":"2024-09-11T01:13:36.502309+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:57.982282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045026199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533379+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:38.533379+0000","last_peered":"2024-09-11T01:13:38.533379+0000","last_clean":"2024-09-11T01:13:38.533379+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:38.533379+0000","last_undegraded":"2024-09-11T01:13:38.533379+0000","last_fullsized":"2024-09-11T01:13:38.533379+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533195+0000","last_change":"2024-09-11T01:13:38.533195+0000","last_active":"2024-09-11T01:13:38.533195+0000","last_peered":"2024-09-11T01:13:38.533195+0000","last_clean":"2024-09-11T01:13:38.533195+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:38.533195+0000","last_undegraded":"2024-09-11T01:13:38.533195+0000","last_fullsized":"2024-09-11T01:13:38.533195+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:36:42.819574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.536120+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:38.536120+0000","last_peered":"2024-09-11T01:13:38.536120+0000","last_clean":"2024-09-11T01:13:38.536120+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:38.536120+0000","last_undegraded":"2024-09-11T01:13:38.536120+0000","last_fullsized":"2024-09-11T01:13:38.536120+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502294+0000","last_became_peered":"2024-09-11T01:13:36.502294+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:19.913634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043864000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533542+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:38.533542+0000","last_peered":"2024-09-11T01:13:38.533542+0000","last_clean":"2024-09-11T01:13:38.533542+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:38.533542+0000","last_undegraded":"2024-09-11T01:13:38.533542+0000","last_fullsized":"2024-09-11T01:13:38.533542+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":29237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:42.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:42.375 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:42.376 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:42.388 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-11T01:13:40.398095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85616,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029904,"statfs":{"total":289910292480,"available":289822621696,"internally_reserved":0,"allocated":376832,"data_stored":87749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":87223632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533864+0000","last_change":"2024-09-11T01:13:38.533864+0000","last_active":"2024-09-11T01:13:38.533864+0000","last_peered":"2024-09-11T01:13:38.533864+0000","last_clean":"2024-09-11T01:13:38.533864+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:38.533864+0000","last_undegraded":"2024-09-11T01:13:38.533864+0000","last_fullsized":"2024-09-11T01:13:38.533864+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:21.523374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.534017+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:38.534017+0000","last_peered":"2024-09-11T01:13:38.534017+0000","last_clean":"2024-09-11T01:13:38.534017+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:38.534017+0000","last_undegraded":"2024-09-11T01:13:38.534017+0000","last_fullsized":"2024-09-11T01:13:38.534017+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502309+0000","last_became_peered":"2024-09-11T01:13:36.502309+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:57.982282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045026199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533379+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:38.533379+0000","last_peered":"2024-09-11T01:13:38.533379+0000","last_clean":"2024-09-11T01:13:38.533379+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:38.533379+0000","last_undegraded":"2024-09-11T01:13:38.533379+0000","last_fullsized":"2024-09-11T01:13:38.533379+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533195+0000","last_change":"2024-09-11T01:13:38.533195+0000","last_active":"2024-09-11T01:13:38.533195+0000","last_peered":"2024-09-11T01:13:38.533195+0000","last_clean":"2024-09-11T01:13:38.533195+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:38.533195+0000","last_undegraded":"2024-09-11T01:13:38.533195+0000","last_fullsized":"2024-09-11T01:13:38.533195+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:36:42.819574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.536120+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:38.536120+0000","last_peered":"2024-09-11T01:13:38.536120+0000","last_clean":"2024-09-11T01:13:38.536120+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:38.536120+0000","last_undegraded":"2024-09-11T01:13:38.536120+0000","last_fullsized":"2024-09-11T01:13:38.536120+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502294+0000","last_became_peered":"2024-09-11T01:13:36.502294+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:19.913634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043864000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533542+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:38.533542+0000","last_peered":"2024-09-11T01:13:38.533542+0000","last_clean":"2024-09-11T01:13:38.533542+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:38.533542+0000","last_undegraded":"2024-09-11T01:13:38.533542+0000","last_fullsized":"2024-09-11T01:13:38.533542+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":29237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:42.389 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T01:13:42.389 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T01:13:42.390 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T01:13:42.390 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T01:13:42.729 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:42.729 INFO:teuthology.orchestra.run.smithi039.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T01:13:42.742 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T01:13:42.743 INFO:teuthology.run_tasks:Running task lost_unfound... 2024-09-11T01:13:42.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:43.092 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:43.092 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":14,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:13:38.508203+0000","last_up_change":"2024-09-11T01:13:29.413330+0000","last_in_change":"2024-09-11T01:13:20.872658+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":7462012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":7462012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":7462012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":7462012}]},"public_addr":"172.21.15.39:6800/7462012","cluster_addr":"172.21.15.39:6801/7462012","heartbeat_back_addr":"172.21.15.39:6803/7462012","heartbeat_front_addr":"172.21.15.39:6802/7462012","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3162293523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":3162293523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":3162293523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3162293523}]},"public_addr":"172.21.15.39:6804/3162293523","cluster_addr":"172.21.15.39:6805/3162293523","heartbeat_back_addr":"172.21.15.39:6807/3162293523","heartbeat_front_addr":"172.21.15.39:6806/3162293523","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":3298933028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":3298933028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":3298933028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3298933028}]},"public_addr":"172.21.15.39:6808/3298933028","cluster_addr":"172.21.15.39:6809/3298933028","heartbeat_back_addr":"172.21.15.39:6811/3298933028","heartbeat_front_addr":"172.21.15.39:6810/3298933028","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:43.105 INFO:tasks.lost_unfound.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-11T01:13:35.284028+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T01:13:43.105 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-11T01:13:43.404 INFO:teuthology.orchestra.run.smithi039.stdout:pg_num: 8 2024-09-11T01:13:43.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-11T01:13:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:epoch 14 2024-09-11T01:13:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:fsid bcb624ec-d02e-4f87-a117-bb3a3b752391 2024-09-11T01:13:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:created 2024-09-11T01:13:19.553161+0000 2024-09-11T01:13:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:modified 2024-09-11T01:13:38.508203+0000 2024-09-11T01:13:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T01:13:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:crush_version 5 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:full_ratio 0.95 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:backfillfull_ratio 0.9 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:nearfull_ratio 0.85 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:require_min_compat_client luminous 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:min_compat_client jewel 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:require_osd_release squid 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:stretch_mode_enabled false 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout: removed_snaps_queue [2~1] 2024-09-11T01:13:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:max_osd 3 2024-09-11T01:13:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:osd.0 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.39:6800/7462012 v2:172.21.15.39:6801/7462012 exists,up f5f0de43-07f1-4db0-9ad7-4564c8fd2953 2024-09-11T01:13:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.39:6804/3162293523 v2:172.21.15.39:6805/3162293523 exists,up 5df6a606-9f1b-4a88-8879-e1661e7956c2 2024-09-11T01:13:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:osd.2 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.39:6808/3298933028 v2:172.21.15.39:6809/3298933028 exists,up ee7baf02-576b-4245-9572-bfd1ab1e3a0d 2024-09-11T01:13:43.728 INFO:tasks.lost_unfound.ceph_manager:['osd.0 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.39:6800/7462012 v2:172.21.15.39:6801/7462012 exists,up f5f0de43-07f1-4db0-9ad7-4564c8fd2953', 'osd.1 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.39:6804/3162293523 v2:172.21.15.39:6805/3162293523 exists,up 5df6a606-9f1b-4a88-8879-e1661e7956c2', 'osd.2 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v2:172.21.15.39:6808/3298933028 v2:172.21.15.39:6809/3298933028 exists,up ee7baf02-576b-4245-9572-bfd1ab1e3a0d'] 2024-09-11T01:13:43.728 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-11T01:13:43.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:44.021 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:44.022 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:44.033 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-11T01:13:42.398351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85616,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029904,"statfs":{"total":289910292480,"available":289822621696,"internally_reserved":0,"allocated":376832,"data_stored":87749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":87223632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000255"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533864+0000","last_change":"2024-09-11T01:13:38.533864+0000","last_active":"2024-09-11T01:13:38.533864+0000","last_peered":"2024-09-11T01:13:38.533864+0000","last_clean":"2024-09-11T01:13:38.533864+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:38.533864+0000","last_undegraded":"2024-09-11T01:13:38.533864+0000","last_fullsized":"2024-09-11T01:13:38.533864+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:21.523374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.534017+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:38.534017+0000","last_peered":"2024-09-11T01:13:38.534017+0000","last_clean":"2024-09-11T01:13:38.534017+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:38.534017+0000","last_undegraded":"2024-09-11T01:13:38.534017+0000","last_fullsized":"2024-09-11T01:13:38.534017+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502309+0000","last_became_peered":"2024-09-11T01:13:36.502309+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:57.982282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045026199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533379+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:38.533379+0000","last_peered":"2024-09-11T01:13:38.533379+0000","last_clean":"2024-09-11T01:13:38.533379+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:38.533379+0000","last_undegraded":"2024-09-11T01:13:38.533379+0000","last_fullsized":"2024-09-11T01:13:38.533379+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533195+0000","last_change":"2024-09-11T01:13:38.533195+0000","last_active":"2024-09-11T01:13:38.533195+0000","last_peered":"2024-09-11T01:13:38.533195+0000","last_clean":"2024-09-11T01:13:38.533195+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:38.533195+0000","last_undegraded":"2024-09-11T01:13:38.533195+0000","last_fullsized":"2024-09-11T01:13:38.533195+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:36:42.819574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.536120+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:38.536120+0000","last_peered":"2024-09-11T01:13:38.536120+0000","last_clean":"2024-09-11T01:13:38.536120+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:38.536120+0000","last_undegraded":"2024-09-11T01:13:38.536120+0000","last_fullsized":"2024-09-11T01:13:38.536120+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502294+0000","last_became_peered":"2024-09-11T01:13:36.502294+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:19.913634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043864000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533542+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:38.533542+0000","last_peered":"2024-09-11T01:13:38.533542+0000","last_clean":"2024-09-11T01:13:38.533542+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:38.533542+0000","last_undegraded":"2024-09-11T01:13:38.533542+0000","last_fullsized":"2024-09-11T01:13:38.533542+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":29237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:44.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:44.325 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:44.326 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:44.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-11T01:13:42.398351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":85616,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":85179,"kb_avail":283029904,"statfs":{"total":289910292480,"available":289822621696,"internally_reserved":0,"allocated":376832,"data_stored":87749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":87223632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000255"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533864+0000","last_change":"2024-09-11T01:13:38.533864+0000","last_active":"2024-09-11T01:13:38.533864+0000","last_peered":"2024-09-11T01:13:38.533864+0000","last_clean":"2024-09-11T01:13:38.533864+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:38.533864+0000","last_undegraded":"2024-09-11T01:13:38.533864+0000","last_fullsized":"2024-09-11T01:13:38.533864+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:21.523374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.534017+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:38.534017+0000","last_peered":"2024-09-11T01:13:38.534017+0000","last_clean":"2024-09-11T01:13:38.534017+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:38.534017+0000","last_undegraded":"2024-09-11T01:13:38.534017+0000","last_fullsized":"2024-09-11T01:13:38.534017+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502309+0000","last_became_peered":"2024-09-11T01:13:36.502309+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:50:57.982282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045026199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533379+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:38.533379+0000","last_peered":"2024-09-11T01:13:38.533379+0000","last_clean":"2024-09-11T01:13:38.533379+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:38.533379+0000","last_undegraded":"2024-09-11T01:13:38.533379+0000","last_fullsized":"2024-09-11T01:13:38.533379+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533195+0000","last_change":"2024-09-11T01:13:38.533195+0000","last_active":"2024-09-11T01:13:38.533195+0000","last_peered":"2024-09-11T01:13:38.533195+0000","last_clean":"2024-09-11T01:13:38.533195+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:38.533195+0000","last_undegraded":"2024-09-11T01:13:38.533195+0000","last_fullsized":"2024-09-11T01:13:38.533195+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:36:42.819574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.536120+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:38.536120+0000","last_peered":"2024-09-11T01:13:38.536120+0000","last_clean":"2024-09-11T01:13:38.536120+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:38.536120+0000","last_undegraded":"2024-09-11T01:13:38.536120+0000","last_fullsized":"2024-09-11T01:13:38.536120+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:39.051820+0000","last_change":"2024-09-11T01:13:39.051820+0000","last_active":"2024-09-11T01:13:39.051820+0000","last_peered":"2024-09-11T01:13:39.051820+0000","last_clean":"2024-09-11T01:13:39.051820+0000","last_became_active":"2024-09-11T01:13:36.502294+0000","last_became_peered":"2024-09-11T01:13:36.502294+0000","last_unstale":"2024-09-11T01:13:39.051820+0000","last_undegraded":"2024-09-11T01:13:39.051820+0000","last_fullsized":"2024-09-11T01:13:39.051820+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:19.913634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043864000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.533542+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:38.533542+0000","last_peered":"2024-09-11T01:13:38.533542+0000","last_clean":"2024-09-11T01:13:38.533542+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:38.533542+0000","last_undegraded":"2024-09-11T01:13:38.533542+0000","last_fullsized":"2024-09-11T01:13:38.533542+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":122880,"data_stored":29237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31316,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340524,"statfs":{"total":96636764160,"available":96604696576,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:44.339 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-11T01:13:44.339 INFO:tasks.lost_unfound.ceph_manager:creating pool_name unfound_pool 2024-09-11T01:13:44.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfound_pool 16 2024-09-11T01:13:45.578 INFO:teuthology.orchestra.run.smithi039.stderr:pool 'unfound_pool' created 2024-09-11T01:13:45.596 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfound_pool rados --yes-i-really-mean-it || true 2024-09-11T01:13:46.598 INFO:teuthology.orchestra.run.smithi039.stderr:enabled application 'rados' on pool 'unfound_pool' 2024-09-11T01:13:47.620 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T01:13:47.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:13:47.660 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T01:13:47.660 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-11T01:13:48.027 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:13:48.025+0000 7f7c4de44640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-11T01:13:48.659 INFO:teuthology.orchestra.run.smithi039.stderr:marked down osd.2. 2024-09-11T01:13:48.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-11T01:13:48.991 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:13:48.988+0000 7f7c4de44640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-11T01:13:49.672 INFO:teuthology.orchestra.run.smithi039.stderr:marked out osd.2. 2024-09-11T01:13:49.691 INFO:tasks.util.rados:rados -p unfound_pool put dummy /etc/fstab 2024-09-11T01:13:49.691 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put dummy /etc/fstab 2024-09-11T01:13:49.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:13:49.748 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:13:49.903 INFO:teuthology.orchestra.run.smithi039.stdout:34359738376 2024-09-11T01:13:49.904 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:13:49.905 INFO:teuthology.orchestra.run.smithi039.stdout:38654705672 2024-09-11T01:13:49.905 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:13:50.263 INFO:teuthology.orchestra.run.smithi039.stdout:38654705670 2024-09-11T01:13:50.263 INFO:teuthology.orchestra.run.smithi039.stdout:34359738374 2024-09-11T01:13:50.277 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738376 got 34359738374 for osd.0 2024-09-11T01:13:50.278 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705670 for osd.1 2024-09-11T01:13:51.278 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:13:51.279 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:13:51.595 INFO:teuthology.orchestra.run.smithi039.stdout:38654705672 2024-09-11T01:13:51.595 INFO:teuthology.orchestra.run.smithi039.stdout:34359738376 2024-09-11T01:13:51.608 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705672 for osd.1 2024-09-11T01:13:51.609 DEBUG:teuthology.parallel:result is None 2024-09-11T01:13:51.609 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738376 got 34359738376 for osd.0 2024-09-11T01:13:51.610 DEBUG:teuthology.parallel:result is None 2024-09-11T01:13:51.610 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-11T01:13:51.610 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:51.906 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:51.906 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:51.918 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-11T01:13:50.678866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":58564,"kb_used_data":336,"kb_used_omap":3,"kb_used_meta":58172,"kb_avail":188685116,"statfs":{"total":193273528320,"available":193213558784,"internally_reserved":0,"allocated":344064,"data_stored":90392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":59569034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.085539"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673118+0000","last_change":"2024-09-11T01:13:46.599645+0000","last_active":"2024-09-11T01:13:49.673118+0000","last_peered":"2024-09-11T01:13:49.673118+0000","last_clean":"2024-09-11T01:13:49.673118+0000","last_became_active":"2024-09-11T01:13:46.599334+0000","last_became_peered":"2024-09-11T01:13:46.599334+0000","last_unstale":"2024-09-11T01:13:49.673118+0000","last_undegraded":"2024-09-11T01:13:49.673118+0000","last_fullsized":"2024-09-11T01:13:49.673118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:55.291272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420270+0000","last_change":"2024-09-11T01:13:48.656228+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420270+0000","last_undegraded":"2024-09-11T01:13:49.420270+0000","last_fullsized":"2024-09-11T01:13:49.420270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426158+0000","last_change":"2024-09-11T01:13:48.658558+0000","last_active":"2024-09-11T01:13:48.658558+0000","last_peered":"2024-09-11T01:13:47.636307+0000","last_clean":"2024-09-11T01:13:47.636307+0000","last_became_active":"2024-09-11T01:13:46.603991+0000","last_became_peered":"2024-09-11T01:13:46.603991+0000","last_unstale":"2024-09-11T01:13:49.426158+0000","last_undegraded":"2024-09-11T01:13:49.426158+0000","last_fullsized":"2024-09-11T01:13:49.426158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420254+0000","last_change":"2024-09-11T01:13:48.657163+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420254+0000","last_undegraded":"2024-09-11T01:13:49.420254+0000","last_fullsized":"2024-09-11T01:13:49.420254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420246+0000","last_change":"2024-09-11T01:13:48.655548+0000","last_active":"2024-09-11T01:13:48.655548+0000","last_peered":"2024-09-11T01:13:48.655003+0000","last_clean":"2024-09-11T01:13:48.655003+0000","last_became_active":"2024-09-11T01:13:46.602323+0000","last_became_peered":"2024-09-11T01:13:46.602323+0000","last_unstale":"2024-09-11T01:13:49.420246+0000","last_undegraded":"2024-09-11T01:13:49.420246+0000","last_fullsized":"2024-09-11T01:13:49.420246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673744+0000","last_change":"2024-09-11T01:13:46.602152+0000","last_active":"2024-09-11T01:13:49.673744+0000","last_peered":"2024-09-11T01:13:49.673744+0000","last_clean":"2024-09-11T01:13:49.673744+0000","last_became_active":"2024-09-11T01:13:46.601872+0000","last_became_peered":"2024-09-11T01:13:46.601872+0000","last_unstale":"2024-09-11T01:13:49.673744+0000","last_undegraded":"2024-09-11T01:13:49.673744+0000","last_fullsized":"2024-09-11T01:13:49.673744+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:08.777062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674628+0000","last_change":"2024-09-11T01:13:46.602657+0000","last_active":"2024-09-11T01:13:49.674628+0000","last_peered":"2024-09-11T01:13:49.674628+0000","last_clean":"2024-09-11T01:13:49.674628+0000","last_became_active":"2024-09-11T01:13:46.600584+0000","last_became_peered":"2024-09-11T01:13:46.600584+0000","last_unstale":"2024-09-11T01:13:49.674628+0000","last_undegraded":"2024-09-11T01:13:49.674628+0000","last_fullsized":"2024-09-11T01:13:49.674628+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:17.638915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420194+0000","last_change":"2024-09-11T01:13:48.656334+0000","last_active":"2024-09-11T01:13:48.656334+0000","last_peered":"2024-09-11T01:13:48.655841+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:13:46.602821+0000","last_became_peered":"2024-09-11T01:13:46.602821+0000","last_unstale":"2024-09-11T01:13:49.420194+0000","last_undegraded":"2024-09-11T01:13:49.420194+0000","last_fullsized":"2024-09-11T01:13:49.420194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674239+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:49.674239+0000","last_peered":"2024-09-11T01:13:49.674239+0000","last_clean":"2024-09-11T01:13:49.674239+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:49.674239+0000","last_undegraded":"2024-09-11T01:13:49.674239+0000","last_fullsized":"2024-09-11T01:13:49.674239+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426122+0000","last_change":"2024-09-11T01:13:48.655644+0000","last_active":"2024-09-11T01:13:48.655644+0000","last_peered":"2024-09-11T01:13:47.636145+0000","last_clean":"2024-09-11T01:13:47.636145+0000","last_became_active":"2024-09-11T01:13:46.604332+0000","last_became_peered":"2024-09-11T01:13:46.604332+0000","last_unstale":"2024-09-11T01:13:49.426122+0000","last_undegraded":"2024-09-11T01:13:49.426122+0000","last_fullsized":"2024-09-11T01:13:49.426122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420213+0000","last_change":"2024-09-11T01:13:48.655430+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420213+0000","last_undegraded":"2024-09-11T01:13:49.420213+0000","last_fullsized":"2024-09-11T01:13:49.420213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672023+0000","last_change":"2024-09-11T01:13:46.599819+0000","last_active":"2024-09-11T01:13:49.672023+0000","last_peered":"2024-09-11T01:13:49.672023+0000","last_clean":"2024-09-11T01:13:49.672023+0000","last_became_active":"2024-09-11T01:13:46.599526+0000","last_became_peered":"2024-09-11T01:13:46.599526+0000","last_unstale":"2024-09-11T01:13:49.672023+0000","last_undegraded":"2024-09-11T01:13:49.672023+0000","last_fullsized":"2024-09-11T01:13:49.672023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:35.784033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672417+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:49.672417+0000","last_peered":"2024-09-11T01:13:49.672417+0000","last_clean":"2024-09-11T01:13:49.672417+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:49.672417+0000","last_undegraded":"2024-09-11T01:13:49.672417+0000","last_fullsized":"2024-09-11T01:13:49.672417+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426076+0000","last_change":"2024-09-11T01:13:48.657650+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426076+0000","last_undegraded":"2024-09-11T01:13:49.426076+0000","last_fullsized":"2024-09-11T01:13:49.426076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.656306+0000","last_change":"2024-09-11T01:13:48.656306+0000","last_active":"2024-09-11T01:13:48.656306+0000","last_peered":"2024-09-11T01:13:47.636183+0000","last_clean":"2024-09-11T01:13:47.636183+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:48.656306+0000","last_undegraded":"2024-09-11T01:13:48.656306+0000","last_fullsized":"2024-09-11T01:13:48.656306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426049+0000","last_change":"2024-09-11T01:13:48.657606+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426049+0000","last_undegraded":"2024-09-11T01:13:49.426049+0000","last_fullsized":"2024-09-11T01:13:49.426049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675542+0000","last_change":"2024-09-11T01:13:46.602073+0000","last_active":"2024-09-11T01:13:49.675542+0000","last_peered":"2024-09-11T01:13:49.675542+0000","last_clean":"2024-09-11T01:13:49.675542+0000","last_became_active":"2024-09-11T01:13:46.601706+0000","last_became_peered":"2024-09-11T01:13:46.601706+0000","last_unstale":"2024-09-11T01:13:49.675542+0000","last_undegraded":"2024-09-11T01:13:49.675542+0000","last_fullsized":"2024-09-11T01:13:49.675542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:15.458280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674952+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:49.674952+0000","last_peered":"2024-09-11T01:13:49.674952+0000","last_clean":"2024-09-11T01:13:49.674952+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:49.674952+0000","last_undegraded":"2024-09-11T01:13:49.674952+0000","last_fullsized":"2024-09-11T01:13:49.674952+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675197+0000","last_change":"2024-09-11T01:13:46.602379+0000","last_active":"2024-09-11T01:13:49.675197+0000","last_peered":"2024-09-11T01:13:49.675197+0000","last_clean":"2024-09-11T01:13:49.675197+0000","last_became_active":"2024-09-11T01:13:46.601931+0000","last_became_peered":"2024-09-11T01:13:46.601931+0000","last_unstale":"2024-09-11T01:13:49.675197+0000","last_undegraded":"2024-09-11T01:13:49.675197+0000","last_fullsized":"2024-09-11T01:13:49.675197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:25.833664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420263+0000","last_change":"2024-09-11T01:13:48.655557+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420263+0000","last_undegraded":"2024-09-11T01:13:49.420263+0000","last_fullsized":"2024-09-11T01:13:49.420263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675364+0000","last_change":"2024-09-11T01:13:46.600876+0000","last_active":"2024-09-11T01:13:49.675364+0000","last_peered":"2024-09-11T01:13:49.675364+0000","last_clean":"2024-09-11T01:13:49.675364+0000","last_became_active":"2024-09-11T01:13:46.600623+0000","last_became_peered":"2024-09-11T01:13:46.600623+0000","last_unstale":"2024-09-11T01:13:49.675364+0000","last_undegraded":"2024-09-11T01:13:49.675364+0000","last_fullsized":"2024-09-11T01:13:49.675364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:10.864417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675310+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:49.675310+0000","last_peered":"2024-09-11T01:13:49.675310+0000","last_clean":"2024-09-11T01:13:49.675310+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:49.675310+0000","last_undegraded":"2024-09-11T01:13:49.675310+0000","last_fullsized":"2024-09-11T01:13:49.675310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.657544+0000","last_change":"2024-09-11T01:13:48.657544+0000","last_active":"2024-09-11T01:13:48.657544+0000","last_peered":"2024-09-11T01:13:47.635950+0000","last_clean":"2024-09-11T01:13:47.635950+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:48.657544+0000","last_undegraded":"2024-09-11T01:13:48.657544+0000","last_fullsized":"2024-09-11T01:13:48.657544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674895+0000","last_change":"2024-09-11T01:13:46.600813+0000","last_active":"2024-09-11T01:13:49.674895+0000","last_peered":"2024-09-11T01:13:49.674895+0000","last_clean":"2024-09-11T01:13:49.674895+0000","last_became_active":"2024-09-11T01:13:46.600533+0000","last_became_peered":"2024-09-11T01:13:46.600533+0000","last_unstale":"2024-09-11T01:13:49.674895+0000","last_undegraded":"2024-09-11T01:13:49.674895+0000","last_fullsized":"2024-09-11T01:13:49.674895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:39.388587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":46582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":167936,"data_stored":43810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:51.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:52.213 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:52.214 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:52.225 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-11T01:13:50.678866+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":58564,"kb_used_data":336,"kb_used_omap":3,"kb_used_meta":58172,"kb_avail":188685116,"statfs":{"total":193273528320,"available":193213558784,"internally_reserved":0,"allocated":344064,"data_stored":90392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":59569034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.085539"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673118+0000","last_change":"2024-09-11T01:13:46.599645+0000","last_active":"2024-09-11T01:13:49.673118+0000","last_peered":"2024-09-11T01:13:49.673118+0000","last_clean":"2024-09-11T01:13:49.673118+0000","last_became_active":"2024-09-11T01:13:46.599334+0000","last_became_peered":"2024-09-11T01:13:46.599334+0000","last_unstale":"2024-09-11T01:13:49.673118+0000","last_undegraded":"2024-09-11T01:13:49.673118+0000","last_fullsized":"2024-09-11T01:13:49.673118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:55.291272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420270+0000","last_change":"2024-09-11T01:13:48.656228+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420270+0000","last_undegraded":"2024-09-11T01:13:49.420270+0000","last_fullsized":"2024-09-11T01:13:49.420270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426158+0000","last_change":"2024-09-11T01:13:48.658558+0000","last_active":"2024-09-11T01:13:48.658558+0000","last_peered":"2024-09-11T01:13:47.636307+0000","last_clean":"2024-09-11T01:13:47.636307+0000","last_became_active":"2024-09-11T01:13:46.603991+0000","last_became_peered":"2024-09-11T01:13:46.603991+0000","last_unstale":"2024-09-11T01:13:49.426158+0000","last_undegraded":"2024-09-11T01:13:49.426158+0000","last_fullsized":"2024-09-11T01:13:49.426158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420254+0000","last_change":"2024-09-11T01:13:48.657163+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420254+0000","last_undegraded":"2024-09-11T01:13:49.420254+0000","last_fullsized":"2024-09-11T01:13:49.420254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420246+0000","last_change":"2024-09-11T01:13:48.655548+0000","last_active":"2024-09-11T01:13:48.655548+0000","last_peered":"2024-09-11T01:13:48.655003+0000","last_clean":"2024-09-11T01:13:48.655003+0000","last_became_active":"2024-09-11T01:13:46.602323+0000","last_became_peered":"2024-09-11T01:13:46.602323+0000","last_unstale":"2024-09-11T01:13:49.420246+0000","last_undegraded":"2024-09-11T01:13:49.420246+0000","last_fullsized":"2024-09-11T01:13:49.420246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673744+0000","last_change":"2024-09-11T01:13:46.602152+0000","last_active":"2024-09-11T01:13:49.673744+0000","last_peered":"2024-09-11T01:13:49.673744+0000","last_clean":"2024-09-11T01:13:49.673744+0000","last_became_active":"2024-09-11T01:13:46.601872+0000","last_became_peered":"2024-09-11T01:13:46.601872+0000","last_unstale":"2024-09-11T01:13:49.673744+0000","last_undegraded":"2024-09-11T01:13:49.673744+0000","last_fullsized":"2024-09-11T01:13:49.673744+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:08.777062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674628+0000","last_change":"2024-09-11T01:13:46.602657+0000","last_active":"2024-09-11T01:13:49.674628+0000","last_peered":"2024-09-11T01:13:49.674628+0000","last_clean":"2024-09-11T01:13:49.674628+0000","last_became_active":"2024-09-11T01:13:46.600584+0000","last_became_peered":"2024-09-11T01:13:46.600584+0000","last_unstale":"2024-09-11T01:13:49.674628+0000","last_undegraded":"2024-09-11T01:13:49.674628+0000","last_fullsized":"2024-09-11T01:13:49.674628+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:17.638915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420194+0000","last_change":"2024-09-11T01:13:48.656334+0000","last_active":"2024-09-11T01:13:48.656334+0000","last_peered":"2024-09-11T01:13:48.655841+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:13:46.602821+0000","last_became_peered":"2024-09-11T01:13:46.602821+0000","last_unstale":"2024-09-11T01:13:49.420194+0000","last_undegraded":"2024-09-11T01:13:49.420194+0000","last_fullsized":"2024-09-11T01:13:49.420194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674239+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:49.674239+0000","last_peered":"2024-09-11T01:13:49.674239+0000","last_clean":"2024-09-11T01:13:49.674239+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:49.674239+0000","last_undegraded":"2024-09-11T01:13:49.674239+0000","last_fullsized":"2024-09-11T01:13:49.674239+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426122+0000","last_change":"2024-09-11T01:13:48.655644+0000","last_active":"2024-09-11T01:13:48.655644+0000","last_peered":"2024-09-11T01:13:47.636145+0000","last_clean":"2024-09-11T01:13:47.636145+0000","last_became_active":"2024-09-11T01:13:46.604332+0000","last_became_peered":"2024-09-11T01:13:46.604332+0000","last_unstale":"2024-09-11T01:13:49.426122+0000","last_undegraded":"2024-09-11T01:13:49.426122+0000","last_fullsized":"2024-09-11T01:13:49.426122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420213+0000","last_change":"2024-09-11T01:13:48.655430+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420213+0000","last_undegraded":"2024-09-11T01:13:49.420213+0000","last_fullsized":"2024-09-11T01:13:49.420213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672023+0000","last_change":"2024-09-11T01:13:46.599819+0000","last_active":"2024-09-11T01:13:49.672023+0000","last_peered":"2024-09-11T01:13:49.672023+0000","last_clean":"2024-09-11T01:13:49.672023+0000","last_became_active":"2024-09-11T01:13:46.599526+0000","last_became_peered":"2024-09-11T01:13:46.599526+0000","last_unstale":"2024-09-11T01:13:49.672023+0000","last_undegraded":"2024-09-11T01:13:49.672023+0000","last_fullsized":"2024-09-11T01:13:49.672023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:35.784033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672417+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:49.672417+0000","last_peered":"2024-09-11T01:13:49.672417+0000","last_clean":"2024-09-11T01:13:49.672417+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:49.672417+0000","last_undegraded":"2024-09-11T01:13:49.672417+0000","last_fullsized":"2024-09-11T01:13:49.672417+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426076+0000","last_change":"2024-09-11T01:13:48.657650+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426076+0000","last_undegraded":"2024-09-11T01:13:49.426076+0000","last_fullsized":"2024-09-11T01:13:49.426076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.656306+0000","last_change":"2024-09-11T01:13:48.656306+0000","last_active":"2024-09-11T01:13:48.656306+0000","last_peered":"2024-09-11T01:13:47.636183+0000","last_clean":"2024-09-11T01:13:47.636183+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:48.656306+0000","last_undegraded":"2024-09-11T01:13:48.656306+0000","last_fullsized":"2024-09-11T01:13:48.656306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426049+0000","last_change":"2024-09-11T01:13:48.657606+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426049+0000","last_undegraded":"2024-09-11T01:13:49.426049+0000","last_fullsized":"2024-09-11T01:13:49.426049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675542+0000","last_change":"2024-09-11T01:13:46.602073+0000","last_active":"2024-09-11T01:13:49.675542+0000","last_peered":"2024-09-11T01:13:49.675542+0000","last_clean":"2024-09-11T01:13:49.675542+0000","last_became_active":"2024-09-11T01:13:46.601706+0000","last_became_peered":"2024-09-11T01:13:46.601706+0000","last_unstale":"2024-09-11T01:13:49.675542+0000","last_undegraded":"2024-09-11T01:13:49.675542+0000","last_fullsized":"2024-09-11T01:13:49.675542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:15.458280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674952+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:49.674952+0000","last_peered":"2024-09-11T01:13:49.674952+0000","last_clean":"2024-09-11T01:13:49.674952+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:49.674952+0000","last_undegraded":"2024-09-11T01:13:49.674952+0000","last_fullsized":"2024-09-11T01:13:49.674952+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675197+0000","last_change":"2024-09-11T01:13:46.602379+0000","last_active":"2024-09-11T01:13:49.675197+0000","last_peered":"2024-09-11T01:13:49.675197+0000","last_clean":"2024-09-11T01:13:49.675197+0000","last_became_active":"2024-09-11T01:13:46.601931+0000","last_became_peered":"2024-09-11T01:13:46.601931+0000","last_unstale":"2024-09-11T01:13:49.675197+0000","last_undegraded":"2024-09-11T01:13:49.675197+0000","last_fullsized":"2024-09-11T01:13:49.675197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:25.833664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420263+0000","last_change":"2024-09-11T01:13:48.655557+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420263+0000","last_undegraded":"2024-09-11T01:13:49.420263+0000","last_fullsized":"2024-09-11T01:13:49.420263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675364+0000","last_change":"2024-09-11T01:13:46.600876+0000","last_active":"2024-09-11T01:13:49.675364+0000","last_peered":"2024-09-11T01:13:49.675364+0000","last_clean":"2024-09-11T01:13:49.675364+0000","last_became_active":"2024-09-11T01:13:46.600623+0000","last_became_peered":"2024-09-11T01:13:46.600623+0000","last_unstale":"2024-09-11T01:13:49.675364+0000","last_undegraded":"2024-09-11T01:13:49.675364+0000","last_fullsized":"2024-09-11T01:13:49.675364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:10.864417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675310+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:49.675310+0000","last_peered":"2024-09-11T01:13:49.675310+0000","last_clean":"2024-09-11T01:13:49.675310+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:49.675310+0000","last_undegraded":"2024-09-11T01:13:49.675310+0000","last_fullsized":"2024-09-11T01:13:49.675310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.657544+0000","last_change":"2024-09-11T01:13:48.657544+0000","last_active":"2024-09-11T01:13:48.657544+0000","last_peered":"2024-09-11T01:13:47.635950+0000","last_clean":"2024-09-11T01:13:47.635950+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:48.657544+0000","last_undegraded":"2024-09-11T01:13:48.657544+0000","last_fullsized":"2024-09-11T01:13:48.657544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674895+0000","last_change":"2024-09-11T01:13:46.600813+0000","last_active":"2024-09-11T01:13:49.674895+0000","last_peered":"2024-09-11T01:13:49.674895+0000","last_clean":"2024-09-11T01:13:49.674895+0000","last_became_active":"2024-09-11T01:13:46.600533+0000","last_became_peered":"2024-09-11T01:13:46.600533+0000","last_unstale":"2024-09-11T01:13:49.674895+0000","last_undegraded":"2024-09-11T01:13:49.674895+0000","last_fullsized":"2024-09-11T01:13:49.674895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:39.388587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":46582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":167936,"data_stored":43810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:52.228 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:52.521 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:52.558 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:52.559 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-11T01:13:52.399820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":58564,"kb_used_data":336,"kb_used_omap":3,"kb_used_meta":58172,"kb_avail":188685116,"statfs":{"total":193273528320,"available":193213558784,"internally_reserved":0,"allocated":344064,"data_stored":90392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":59569034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.765955"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673118+0000","last_change":"2024-09-11T01:13:46.599645+0000","last_active":"2024-09-11T01:13:49.673118+0000","last_peered":"2024-09-11T01:13:49.673118+0000","last_clean":"2024-09-11T01:13:49.673118+0000","last_became_active":"2024-09-11T01:13:46.599334+0000","last_became_peered":"2024-09-11T01:13:46.599334+0000","last_unstale":"2024-09-11T01:13:49.673118+0000","last_undegraded":"2024-09-11T01:13:49.673118+0000","last_fullsized":"2024-09-11T01:13:49.673118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:55.291272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420270+0000","last_change":"2024-09-11T01:13:48.656228+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420270+0000","last_undegraded":"2024-09-11T01:13:49.420270+0000","last_fullsized":"2024-09-11T01:13:49.420270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426158+0000","last_change":"2024-09-11T01:13:48.658558+0000","last_active":"2024-09-11T01:13:48.658558+0000","last_peered":"2024-09-11T01:13:47.636307+0000","last_clean":"2024-09-11T01:13:47.636307+0000","last_became_active":"2024-09-11T01:13:46.603991+0000","last_became_peered":"2024-09-11T01:13:46.603991+0000","last_unstale":"2024-09-11T01:13:49.426158+0000","last_undegraded":"2024-09-11T01:13:49.426158+0000","last_fullsized":"2024-09-11T01:13:49.426158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420254+0000","last_change":"2024-09-11T01:13:48.657163+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420254+0000","last_undegraded":"2024-09-11T01:13:49.420254+0000","last_fullsized":"2024-09-11T01:13:49.420254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420246+0000","last_change":"2024-09-11T01:13:48.655548+0000","last_active":"2024-09-11T01:13:48.655548+0000","last_peered":"2024-09-11T01:13:48.655003+0000","last_clean":"2024-09-11T01:13:48.655003+0000","last_became_active":"2024-09-11T01:13:46.602323+0000","last_became_peered":"2024-09-11T01:13:46.602323+0000","last_unstale":"2024-09-11T01:13:49.420246+0000","last_undegraded":"2024-09-11T01:13:49.420246+0000","last_fullsized":"2024-09-11T01:13:49.420246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673744+0000","last_change":"2024-09-11T01:13:46.602152+0000","last_active":"2024-09-11T01:13:49.673744+0000","last_peered":"2024-09-11T01:13:49.673744+0000","last_clean":"2024-09-11T01:13:49.673744+0000","last_became_active":"2024-09-11T01:13:46.601872+0000","last_became_peered":"2024-09-11T01:13:46.601872+0000","last_unstale":"2024-09-11T01:13:49.673744+0000","last_undegraded":"2024-09-11T01:13:49.673744+0000","last_fullsized":"2024-09-11T01:13:49.673744+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:08.777062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674628+0000","last_change":"2024-09-11T01:13:46.602657+0000","last_active":"2024-09-11T01:13:49.674628+0000","last_peered":"2024-09-11T01:13:49.674628+0000","last_clean":"2024-09-11T01:13:49.674628+0000","last_became_active":"2024-09-11T01:13:46.600584+0000","last_became_peered":"2024-09-11T01:13:46.600584+0000","last_unstale":"2024-09-11T01:13:49.674628+0000","last_undegraded":"2024-09-11T01:13:49.674628+0000","last_fullsized":"2024-09-11T01:13:49.674628+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:17.638915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420194+0000","last_change":"2024-09-11T01:13:48.656334+0000","last_active":"2024-09-11T01:13:48.656334+0000","last_peered":"2024-09-11T01:13:48.655841+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:13:46.602821+0000","last_became_peered":"2024-09-11T01:13:46.602821+0000","last_unstale":"2024-09-11T01:13:49.420194+0000","last_undegraded":"2024-09-11T01:13:49.420194+0000","last_fullsized":"2024-09-11T01:13:49.420194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674239+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:49.674239+0000","last_peered":"2024-09-11T01:13:49.674239+0000","last_clean":"2024-09-11T01:13:49.674239+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:49.674239+0000","last_undegraded":"2024-09-11T01:13:49.674239+0000","last_fullsized":"2024-09-11T01:13:49.674239+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426122+0000","last_change":"2024-09-11T01:13:48.655644+0000","last_active":"2024-09-11T01:13:48.655644+0000","last_peered":"2024-09-11T01:13:47.636145+0000","last_clean":"2024-09-11T01:13:47.636145+0000","last_became_active":"2024-09-11T01:13:46.604332+0000","last_became_peered":"2024-09-11T01:13:46.604332+0000","last_unstale":"2024-09-11T01:13:49.426122+0000","last_undegraded":"2024-09-11T01:13:49.426122+0000","last_fullsized":"2024-09-11T01:13:49.426122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420213+0000","last_change":"2024-09-11T01:13:48.655430+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420213+0000","last_undegraded":"2024-09-11T01:13:49.420213+0000","last_fullsized":"2024-09-11T01:13:49.420213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672023+0000","last_change":"2024-09-11T01:13:46.599819+0000","last_active":"2024-09-11T01:13:49.672023+0000","last_peered":"2024-09-11T01:13:49.672023+0000","last_clean":"2024-09-11T01:13:49.672023+0000","last_became_active":"2024-09-11T01:13:46.599526+0000","last_became_peered":"2024-09-11T01:13:46.599526+0000","last_unstale":"2024-09-11T01:13:49.672023+0000","last_undegraded":"2024-09-11T01:13:49.672023+0000","last_fullsized":"2024-09-11T01:13:49.672023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:35.784033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672417+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:49.672417+0000","last_peered":"2024-09-11T01:13:49.672417+0000","last_clean":"2024-09-11T01:13:49.672417+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:49.672417+0000","last_undegraded":"2024-09-11T01:13:49.672417+0000","last_fullsized":"2024-09-11T01:13:49.672417+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426076+0000","last_change":"2024-09-11T01:13:48.657650+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426076+0000","last_undegraded":"2024-09-11T01:13:49.426076+0000","last_fullsized":"2024-09-11T01:13:49.426076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.656306+0000","last_change":"2024-09-11T01:13:48.656306+0000","last_active":"2024-09-11T01:13:48.656306+0000","last_peered":"2024-09-11T01:13:47.636183+0000","last_clean":"2024-09-11T01:13:47.636183+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:48.656306+0000","last_undegraded":"2024-09-11T01:13:48.656306+0000","last_fullsized":"2024-09-11T01:13:48.656306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426049+0000","last_change":"2024-09-11T01:13:48.657606+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426049+0000","last_undegraded":"2024-09-11T01:13:49.426049+0000","last_fullsized":"2024-09-11T01:13:49.426049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675542+0000","last_change":"2024-09-11T01:13:46.602073+0000","last_active":"2024-09-11T01:13:49.675542+0000","last_peered":"2024-09-11T01:13:49.675542+0000","last_clean":"2024-09-11T01:13:49.675542+0000","last_became_active":"2024-09-11T01:13:46.601706+0000","last_became_peered":"2024-09-11T01:13:46.601706+0000","last_unstale":"2024-09-11T01:13:49.675542+0000","last_undegraded":"2024-09-11T01:13:49.675542+0000","last_fullsized":"2024-09-11T01:13:49.675542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:15.458280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674952+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:49.674952+0000","last_peered":"2024-09-11T01:13:49.674952+0000","last_clean":"2024-09-11T01:13:49.674952+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:49.674952+0000","last_undegraded":"2024-09-11T01:13:49.674952+0000","last_fullsized":"2024-09-11T01:13:49.674952+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675197+0000","last_change":"2024-09-11T01:13:46.602379+0000","last_active":"2024-09-11T01:13:49.675197+0000","last_peered":"2024-09-11T01:13:49.675197+0000","last_clean":"2024-09-11T01:13:49.675197+0000","last_became_active":"2024-09-11T01:13:46.601931+0000","last_became_peered":"2024-09-11T01:13:46.601931+0000","last_unstale":"2024-09-11T01:13:49.675197+0000","last_undegraded":"2024-09-11T01:13:49.675197+0000","last_fullsized":"2024-09-11T01:13:49.675197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:25.833664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420263+0000","last_change":"2024-09-11T01:13:48.655557+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420263+0000","last_undegraded":"2024-09-11T01:13:49.420263+0000","last_fullsized":"2024-09-11T01:13:49.420263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675364+0000","last_change":"2024-09-11T01:13:46.600876+0000","last_active":"2024-09-11T01:13:49.675364+0000","last_peered":"2024-09-11T01:13:49.675364+0000","last_clean":"2024-09-11T01:13:49.675364+0000","last_became_active":"2024-09-11T01:13:46.600623+0000","last_became_peered":"2024-09-11T01:13:46.600623+0000","last_unstale":"2024-09-11T01:13:49.675364+0000","last_undegraded":"2024-09-11T01:13:49.675364+0000","last_fullsized":"2024-09-11T01:13:49.675364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:10.864417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675310+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:49.675310+0000","last_peered":"2024-09-11T01:13:49.675310+0000","last_clean":"2024-09-11T01:13:49.675310+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:49.675310+0000","last_undegraded":"2024-09-11T01:13:49.675310+0000","last_fullsized":"2024-09-11T01:13:49.675310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.657544+0000","last_change":"2024-09-11T01:13:48.657544+0000","last_active":"2024-09-11T01:13:48.657544+0000","last_peered":"2024-09-11T01:13:47.635950+0000","last_clean":"2024-09-11T01:13:47.635950+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:48.657544+0000","last_undegraded":"2024-09-11T01:13:48.657544+0000","last_fullsized":"2024-09-11T01:13:48.657544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674895+0000","last_change":"2024-09-11T01:13:46.600813+0000","last_active":"2024-09-11T01:13:49.674895+0000","last_peered":"2024-09-11T01:13:49.674895+0000","last_clean":"2024-09-11T01:13:49.674895+0000","last_became_active":"2024-09-11T01:13:46.600533+0000","last_became_peered":"2024-09-11T01:13:46.600533+0000","last_unstale":"2024-09-11T01:13:49.674895+0000","last_undegraded":"2024-09-11T01:13:49.674895+0000","last_fullsized":"2024-09-11T01:13:49.674895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:39.388587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":46582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":167936,"data_stored":43810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:55.561 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:55.872 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:55.873 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:55.885 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-11T01:13:54.400129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":58564,"kb_used_data":336,"kb_used_omap":3,"kb_used_meta":58172,"kb_avail":188685116,"statfs":{"total":193273528320,"available":193213558784,"internally_reserved":0,"allocated":344064,"data_stored":90392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":59569034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000907"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673118+0000","last_change":"2024-09-11T01:13:46.599645+0000","last_active":"2024-09-11T01:13:49.673118+0000","last_peered":"2024-09-11T01:13:49.673118+0000","last_clean":"2024-09-11T01:13:49.673118+0000","last_became_active":"2024-09-11T01:13:46.599334+0000","last_became_peered":"2024-09-11T01:13:46.599334+0000","last_unstale":"2024-09-11T01:13:49.673118+0000","last_undegraded":"2024-09-11T01:13:49.673118+0000","last_fullsized":"2024-09-11T01:13:49.673118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:55.291272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420270+0000","last_change":"2024-09-11T01:13:48.656228+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420270+0000","last_undegraded":"2024-09-11T01:13:49.420270+0000","last_fullsized":"2024-09-11T01:13:49.420270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426158+0000","last_change":"2024-09-11T01:13:48.658558+0000","last_active":"2024-09-11T01:13:48.658558+0000","last_peered":"2024-09-11T01:13:47.636307+0000","last_clean":"2024-09-11T01:13:47.636307+0000","last_became_active":"2024-09-11T01:13:46.603991+0000","last_became_peered":"2024-09-11T01:13:46.603991+0000","last_unstale":"2024-09-11T01:13:49.426158+0000","last_undegraded":"2024-09-11T01:13:49.426158+0000","last_fullsized":"2024-09-11T01:13:49.426158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420254+0000","last_change":"2024-09-11T01:13:48.657163+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420254+0000","last_undegraded":"2024-09-11T01:13:49.420254+0000","last_fullsized":"2024-09-11T01:13:49.420254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420246+0000","last_change":"2024-09-11T01:13:48.655548+0000","last_active":"2024-09-11T01:13:48.655548+0000","last_peered":"2024-09-11T01:13:48.655003+0000","last_clean":"2024-09-11T01:13:48.655003+0000","last_became_active":"2024-09-11T01:13:46.602323+0000","last_became_peered":"2024-09-11T01:13:46.602323+0000","last_unstale":"2024-09-11T01:13:49.420246+0000","last_undegraded":"2024-09-11T01:13:49.420246+0000","last_fullsized":"2024-09-11T01:13:49.420246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673744+0000","last_change":"2024-09-11T01:13:46.602152+0000","last_active":"2024-09-11T01:13:49.673744+0000","last_peered":"2024-09-11T01:13:49.673744+0000","last_clean":"2024-09-11T01:13:49.673744+0000","last_became_active":"2024-09-11T01:13:46.601872+0000","last_became_peered":"2024-09-11T01:13:46.601872+0000","last_unstale":"2024-09-11T01:13:49.673744+0000","last_undegraded":"2024-09-11T01:13:49.673744+0000","last_fullsized":"2024-09-11T01:13:49.673744+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:08.777062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674628+0000","last_change":"2024-09-11T01:13:46.602657+0000","last_active":"2024-09-11T01:13:49.674628+0000","last_peered":"2024-09-11T01:13:49.674628+0000","last_clean":"2024-09-11T01:13:49.674628+0000","last_became_active":"2024-09-11T01:13:46.600584+0000","last_became_peered":"2024-09-11T01:13:46.600584+0000","last_unstale":"2024-09-11T01:13:49.674628+0000","last_undegraded":"2024-09-11T01:13:49.674628+0000","last_fullsized":"2024-09-11T01:13:49.674628+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:17.638915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420194+0000","last_change":"2024-09-11T01:13:48.656334+0000","last_active":"2024-09-11T01:13:48.656334+0000","last_peered":"2024-09-11T01:13:48.655841+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:13:46.602821+0000","last_became_peered":"2024-09-11T01:13:46.602821+0000","last_unstale":"2024-09-11T01:13:49.420194+0000","last_undegraded":"2024-09-11T01:13:49.420194+0000","last_fullsized":"2024-09-11T01:13:49.420194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674239+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:49.674239+0000","last_peered":"2024-09-11T01:13:49.674239+0000","last_clean":"2024-09-11T01:13:49.674239+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:49.674239+0000","last_undegraded":"2024-09-11T01:13:49.674239+0000","last_fullsized":"2024-09-11T01:13:49.674239+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426122+0000","last_change":"2024-09-11T01:13:48.655644+0000","last_active":"2024-09-11T01:13:48.655644+0000","last_peered":"2024-09-11T01:13:47.636145+0000","last_clean":"2024-09-11T01:13:47.636145+0000","last_became_active":"2024-09-11T01:13:46.604332+0000","last_became_peered":"2024-09-11T01:13:46.604332+0000","last_unstale":"2024-09-11T01:13:49.426122+0000","last_undegraded":"2024-09-11T01:13:49.426122+0000","last_fullsized":"2024-09-11T01:13:49.426122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420213+0000","last_change":"2024-09-11T01:13:48.655430+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420213+0000","last_undegraded":"2024-09-11T01:13:49.420213+0000","last_fullsized":"2024-09-11T01:13:49.420213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672023+0000","last_change":"2024-09-11T01:13:46.599819+0000","last_active":"2024-09-11T01:13:49.672023+0000","last_peered":"2024-09-11T01:13:49.672023+0000","last_clean":"2024-09-11T01:13:49.672023+0000","last_became_active":"2024-09-11T01:13:46.599526+0000","last_became_peered":"2024-09-11T01:13:46.599526+0000","last_unstale":"2024-09-11T01:13:49.672023+0000","last_undegraded":"2024-09-11T01:13:49.672023+0000","last_fullsized":"2024-09-11T01:13:49.672023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:35.784033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672417+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:49.672417+0000","last_peered":"2024-09-11T01:13:49.672417+0000","last_clean":"2024-09-11T01:13:49.672417+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:49.672417+0000","last_undegraded":"2024-09-11T01:13:49.672417+0000","last_fullsized":"2024-09-11T01:13:49.672417+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426076+0000","last_change":"2024-09-11T01:13:48.657650+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426076+0000","last_undegraded":"2024-09-11T01:13:49.426076+0000","last_fullsized":"2024-09-11T01:13:49.426076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.656306+0000","last_change":"2024-09-11T01:13:48.656306+0000","last_active":"2024-09-11T01:13:48.656306+0000","last_peered":"2024-09-11T01:13:47.636183+0000","last_clean":"2024-09-11T01:13:47.636183+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:48.656306+0000","last_undegraded":"2024-09-11T01:13:48.656306+0000","last_fullsized":"2024-09-11T01:13:48.656306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426049+0000","last_change":"2024-09-11T01:13:48.657606+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426049+0000","last_undegraded":"2024-09-11T01:13:49.426049+0000","last_fullsized":"2024-09-11T01:13:49.426049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675542+0000","last_change":"2024-09-11T01:13:46.602073+0000","last_active":"2024-09-11T01:13:49.675542+0000","last_peered":"2024-09-11T01:13:49.675542+0000","last_clean":"2024-09-11T01:13:49.675542+0000","last_became_active":"2024-09-11T01:13:46.601706+0000","last_became_peered":"2024-09-11T01:13:46.601706+0000","last_unstale":"2024-09-11T01:13:49.675542+0000","last_undegraded":"2024-09-11T01:13:49.675542+0000","last_fullsized":"2024-09-11T01:13:49.675542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:15.458280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674952+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:49.674952+0000","last_peered":"2024-09-11T01:13:49.674952+0000","last_clean":"2024-09-11T01:13:49.674952+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:49.674952+0000","last_undegraded":"2024-09-11T01:13:49.674952+0000","last_fullsized":"2024-09-11T01:13:49.674952+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675197+0000","last_change":"2024-09-11T01:13:46.602379+0000","last_active":"2024-09-11T01:13:49.675197+0000","last_peered":"2024-09-11T01:13:49.675197+0000","last_clean":"2024-09-11T01:13:49.675197+0000","last_became_active":"2024-09-11T01:13:46.601931+0000","last_became_peered":"2024-09-11T01:13:46.601931+0000","last_unstale":"2024-09-11T01:13:49.675197+0000","last_undegraded":"2024-09-11T01:13:49.675197+0000","last_fullsized":"2024-09-11T01:13:49.675197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:25.833664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420263+0000","last_change":"2024-09-11T01:13:48.655557+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420263+0000","last_undegraded":"2024-09-11T01:13:49.420263+0000","last_fullsized":"2024-09-11T01:13:49.420263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675364+0000","last_change":"2024-09-11T01:13:46.600876+0000","last_active":"2024-09-11T01:13:49.675364+0000","last_peered":"2024-09-11T01:13:49.675364+0000","last_clean":"2024-09-11T01:13:49.675364+0000","last_became_active":"2024-09-11T01:13:46.600623+0000","last_became_peered":"2024-09-11T01:13:46.600623+0000","last_unstale":"2024-09-11T01:13:49.675364+0000","last_undegraded":"2024-09-11T01:13:49.675364+0000","last_fullsized":"2024-09-11T01:13:49.675364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:10.864417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675310+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:49.675310+0000","last_peered":"2024-09-11T01:13:49.675310+0000","last_clean":"2024-09-11T01:13:49.675310+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:49.675310+0000","last_undegraded":"2024-09-11T01:13:49.675310+0000","last_fullsized":"2024-09-11T01:13:49.675310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.657544+0000","last_change":"2024-09-11T01:13:48.657544+0000","last_active":"2024-09-11T01:13:48.657544+0000","last_peered":"2024-09-11T01:13:47.635950+0000","last_clean":"2024-09-11T01:13:47.635950+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:48.657544+0000","last_undegraded":"2024-09-11T01:13:48.657544+0000","last_fullsized":"2024-09-11T01:13:48.657544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674895+0000","last_change":"2024-09-11T01:13:46.600813+0000","last_active":"2024-09-11T01:13:49.674895+0000","last_peered":"2024-09-11T01:13:49.674895+0000","last_clean":"2024-09-11T01:13:49.674895+0000","last_became_active":"2024-09-11T01:13:46.600533+0000","last_became_peered":"2024-09-11T01:13:46.600533+0000","last_unstale":"2024-09-11T01:13:49.674895+0000","last_undegraded":"2024-09-11T01:13:49.674895+0000","last_fullsized":"2024-09-11T01:13:49.674895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:39.388587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":46582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":167936,"data_stored":43810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:55.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:56.179 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:56.179 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:56.191 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-11T01:13:54.400129+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":58564,"kb_used_data":336,"kb_used_omap":3,"kb_used_meta":58172,"kb_avail":188685116,"statfs":{"total":193273528320,"available":193213558784,"internally_reserved":0,"allocated":344064,"data_stored":90392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":59569034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000907"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673118+0000","last_change":"2024-09-11T01:13:46.599645+0000","last_active":"2024-09-11T01:13:49.673118+0000","last_peered":"2024-09-11T01:13:49.673118+0000","last_clean":"2024-09-11T01:13:49.673118+0000","last_became_active":"2024-09-11T01:13:46.599334+0000","last_became_peered":"2024-09-11T01:13:46.599334+0000","last_unstale":"2024-09-11T01:13:49.673118+0000","last_undegraded":"2024-09-11T01:13:49.673118+0000","last_fullsized":"2024-09-11T01:13:49.673118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:55.291272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420270+0000","last_change":"2024-09-11T01:13:48.656228+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420270+0000","last_undegraded":"2024-09-11T01:13:49.420270+0000","last_fullsized":"2024-09-11T01:13:49.420270+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426158+0000","last_change":"2024-09-11T01:13:48.658558+0000","last_active":"2024-09-11T01:13:48.658558+0000","last_peered":"2024-09-11T01:13:47.636307+0000","last_clean":"2024-09-11T01:13:47.636307+0000","last_became_active":"2024-09-11T01:13:46.603991+0000","last_became_peered":"2024-09-11T01:13:46.603991+0000","last_unstale":"2024-09-11T01:13:49.426158+0000","last_undegraded":"2024-09-11T01:13:49.426158+0000","last_fullsized":"2024-09-11T01:13:49.426158+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420254+0000","last_change":"2024-09-11T01:13:48.657163+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420254+0000","last_undegraded":"2024-09-11T01:13:49.420254+0000","last_fullsized":"2024-09-11T01:13:49.420254+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420246+0000","last_change":"2024-09-11T01:13:48.655548+0000","last_active":"2024-09-11T01:13:48.655548+0000","last_peered":"2024-09-11T01:13:48.655003+0000","last_clean":"2024-09-11T01:13:48.655003+0000","last_became_active":"2024-09-11T01:13:46.602323+0000","last_became_peered":"2024-09-11T01:13:46.602323+0000","last_unstale":"2024-09-11T01:13:49.420246+0000","last_undegraded":"2024-09-11T01:13:49.420246+0000","last_fullsized":"2024-09-11T01:13:49.420246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.673744+0000","last_change":"2024-09-11T01:13:46.602152+0000","last_active":"2024-09-11T01:13:49.673744+0000","last_peered":"2024-09-11T01:13:49.673744+0000","last_clean":"2024-09-11T01:13:49.673744+0000","last_became_active":"2024-09-11T01:13:46.601872+0000","last_became_peered":"2024-09-11T01:13:46.601872+0000","last_unstale":"2024-09-11T01:13:49.673744+0000","last_undegraded":"2024-09-11T01:13:49.673744+0000","last_fullsized":"2024-09-11T01:13:49.673744+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:08.777062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674628+0000","last_change":"2024-09-11T01:13:46.602657+0000","last_active":"2024-09-11T01:13:49.674628+0000","last_peered":"2024-09-11T01:13:49.674628+0000","last_clean":"2024-09-11T01:13:49.674628+0000","last_became_active":"2024-09-11T01:13:46.600584+0000","last_became_peered":"2024-09-11T01:13:46.600584+0000","last_unstale":"2024-09-11T01:13:49.674628+0000","last_undegraded":"2024-09-11T01:13:49.674628+0000","last_fullsized":"2024-09-11T01:13:49.674628+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:17.638915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420194+0000","last_change":"2024-09-11T01:13:48.656334+0000","last_active":"2024-09-11T01:13:48.656334+0000","last_peered":"2024-09-11T01:13:48.655841+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:13:46.602821+0000","last_became_peered":"2024-09-11T01:13:46.602821+0000","last_unstale":"2024-09-11T01:13:49.420194+0000","last_undegraded":"2024-09-11T01:13:49.420194+0000","last_fullsized":"2024-09-11T01:13:49.420194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674239+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:49.674239+0000","last_peered":"2024-09-11T01:13:49.674239+0000","last_clean":"2024-09-11T01:13:49.674239+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:49.674239+0000","last_undegraded":"2024-09-11T01:13:49.674239+0000","last_fullsized":"2024-09-11T01:13:49.674239+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426122+0000","last_change":"2024-09-11T01:13:48.655644+0000","last_active":"2024-09-11T01:13:48.655644+0000","last_peered":"2024-09-11T01:13:47.636145+0000","last_clean":"2024-09-11T01:13:47.636145+0000","last_became_active":"2024-09-11T01:13:46.604332+0000","last_became_peered":"2024-09-11T01:13:46.604332+0000","last_unstale":"2024-09-11T01:13:49.426122+0000","last_undegraded":"2024-09-11T01:13:49.426122+0000","last_fullsized":"2024-09-11T01:13:49.426122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420213+0000","last_change":"2024-09-11T01:13:48.655430+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420213+0000","last_undegraded":"2024-09-11T01:13:49.420213+0000","last_fullsized":"2024-09-11T01:13:49.420213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672023+0000","last_change":"2024-09-11T01:13:46.599819+0000","last_active":"2024-09-11T01:13:49.672023+0000","last_peered":"2024-09-11T01:13:49.672023+0000","last_clean":"2024-09-11T01:13:49.672023+0000","last_became_active":"2024-09-11T01:13:46.599526+0000","last_became_peered":"2024-09-11T01:13:46.599526+0000","last_unstale":"2024-09-11T01:13:49.672023+0000","last_undegraded":"2024-09-11T01:13:49.672023+0000","last_fullsized":"2024-09-11T01:13:49.672023+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:35.784033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":33,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.672417+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:49.672417+0000","last_peered":"2024-09-11T01:13:49.672417+0000","last_clean":"2024-09-11T01:13:49.672417+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:49.672417+0000","last_undegraded":"2024-09-11T01:13:49.672417+0000","last_fullsized":"2024-09-11T01:13:49.672417+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426076+0000","last_change":"2024-09-11T01:13:48.657650+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426076+0000","last_undegraded":"2024-09-11T01:13:49.426076+0000","last_fullsized":"2024-09-11T01:13:49.426076+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.656306+0000","last_change":"2024-09-11T01:13:48.656306+0000","last_active":"2024-09-11T01:13:48.656306+0000","last_peered":"2024-09-11T01:13:47.636183+0000","last_clean":"2024-09-11T01:13:47.636183+0000","last_became_active":"2024-09-11T01:13:36.496934+0000","last_became_peered":"2024-09-11T01:13:36.496934+0000","last_unstale":"2024-09-11T01:13:48.656306+0000","last_undegraded":"2024-09-11T01:13:48.656306+0000","last_fullsized":"2024-09-11T01:13:48.656306+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.426049+0000","last_change":"2024-09-11T01:13:48.657606+0000","last_active":"2024-09-11T01:13:45.564293+0000","last_peered":"2024-09-11T01:13:45.564293+0000","last_clean":"2024-09-11T01:13:45.564293+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.426049+0000","last_undegraded":"2024-09-11T01:13:49.426049+0000","last_fullsized":"2024-09-11T01:13:49.426049+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675542+0000","last_change":"2024-09-11T01:13:46.602073+0000","last_active":"2024-09-11T01:13:49.675542+0000","last_peered":"2024-09-11T01:13:49.675542+0000","last_clean":"2024-09-11T01:13:49.675542+0000","last_became_active":"2024-09-11T01:13:46.601706+0000","last_became_peered":"2024-09-11T01:13:46.601706+0000","last_unstale":"2024-09-11T01:13:49.675542+0000","last_undegraded":"2024-09-11T01:13:49.675542+0000","last_fullsized":"2024-09-11T01:13:49.675542+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:15.458280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674952+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:49.674952+0000","last_peered":"2024-09-11T01:13:49.674952+0000","last_clean":"2024-09-11T01:13:49.674952+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:49.674952+0000","last_undegraded":"2024-09-11T01:13:49.674952+0000","last_fullsized":"2024-09-11T01:13:49.674952+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675197+0000","last_change":"2024-09-11T01:13:46.602379+0000","last_active":"2024-09-11T01:13:49.675197+0000","last_peered":"2024-09-11T01:13:49.675197+0000","last_clean":"2024-09-11T01:13:49.675197+0000","last_became_active":"2024-09-11T01:13:46.601931+0000","last_became_peered":"2024-09-11T01:13:46.601931+0000","last_unstale":"2024-09-11T01:13:49.675197+0000","last_undegraded":"2024-09-11T01:13:49.675197+0000","last_fullsized":"2024-09-11T01:13:49.675197+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:25.833664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:49.420263+0000","last_change":"2024-09-11T01:13:48.655557+0000","last_active":"2024-09-11T01:13:35.453339+0000","last_peered":"2024-09-11T01:13:35.453339+0000","last_clean":"2024-09-11T01:13:35.453339+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:13:49.420263+0000","last_undegraded":"2024-09-11T01:13:49.420263+0000","last_fullsized":"2024-09-11T01:13:49.420263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675364+0000","last_change":"2024-09-11T01:13:46.600876+0000","last_active":"2024-09-11T01:13:49.675364+0000","last_peered":"2024-09-11T01:13:49.675364+0000","last_clean":"2024-09-11T01:13:49.675364+0000","last_became_active":"2024-09-11T01:13:46.600623+0000","last_became_peered":"2024-09-11T01:13:46.600623+0000","last_unstale":"2024-09-11T01:13:49.675364+0000","last_undegraded":"2024-09-11T01:13:49.675364+0000","last_fullsized":"2024-09-11T01:13:49.675364+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:10.864417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.675310+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:49.675310+0000","last_peered":"2024-09-11T01:13:49.675310+0000","last_clean":"2024-09-11T01:13:49.675310+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:49.675310+0000","last_undegraded":"2024-09-11T01:13:49.675310+0000","last_fullsized":"2024-09-11T01:13:49.675310+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:13:48.657544+0000","last_change":"2024-09-11T01:13:48.657544+0000","last_active":"2024-09-11T01:13:48.657544+0000","last_peered":"2024-09-11T01:13:47.635950+0000","last_clean":"2024-09-11T01:13:47.635950+0000","last_became_active":"2024-09-11T01:13:36.497173+0000","last_became_peered":"2024-09-11T01:13:36.497173+0000","last_unstale":"2024-09-11T01:13:48.657544+0000","last_undegraded":"2024-09-11T01:13:48.657544+0000","last_fullsized":"2024-09-11T01:13:48.657544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.674895+0000","last_change":"2024-09-11T01:13:46.600813+0000","last_active":"2024-09-11T01:13:49.674895+0000","last_peered":"2024-09-11T01:13:49.674895+0000","last_clean":"2024-09-11T01:13:49.674895+0000","last_became_active":"2024-09-11T01:13:46.600533+0000","last_became_peered":"2024-09-11T01:13:46.600533+0000","last_unstale":"2024-09-11T01:13:49.674895+0000","last_undegraded":"2024-09-11T01:13:49.674895+0000","last_fullsized":"2024-09-11T01:13:49.674895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:39.388587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":176128,"data_stored":46582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31356,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340484,"statfs":{"total":96636764160,"available":96604655616,"internally_reserved":0,"allocated":167936,"data_stored":43810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:59.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:59.502 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:13:59.502 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:13:59.514 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-11T01:13:58.400862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":58588,"kb_used_data":368,"kb_used_omap":3,"kb_used_meta":58172,"kb_avail":188685092,"statfs":{"total":193273528320,"available":193213534208,"internally_reserved":0,"allocated":376832,"data_stored":99558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":59569034},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.729695"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.679749+0000","last_change":"2024-09-11T01:13:46.599645+0000","last_active":"2024-09-11T01:13:50.679749+0000","last_peered":"2024-09-11T01:13:50.679749+0000","last_clean":"2024-09-11T01:13:50.679749+0000","last_became_active":"2024-09-11T01:13:46.599334+0000","last_became_peered":"2024-09-11T01:13:46.599334+0000","last_unstale":"2024-09-11T01:13:50.679749+0000","last_undegraded":"2024-09-11T01:13:50.679749+0000","last_fullsized":"2024-09-11T01:13:50.679749+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:55.291272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.683761+0000","last_change":"2024-09-11T01:13:50.683687+0000","last_active":"2024-09-11T01:13:50.683761+0000","last_peered":"2024-09-11T01:13:50.683761+0000","last_clean":"2024-09-11T01:13:50.683761+0000","last_became_active":"2024-09-11T01:13:50.683431+0000","last_became_peered":"2024-09-11T01:13:50.683431+0000","last_unstale":"2024-09-11T01:13:50.683761+0000","last_undegraded":"2024-09-11T01:13:50.683761+0000","last_fullsized":"2024-09-11T01:13:50.683761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:11.732084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.686087+0000","last_change":"2024-09-11T01:13:50.686011+0000","last_active":"2024-09-11T01:13:50.686087+0000","last_peered":"2024-09-11T01:13:50.686087+0000","last_clean":"2024-09-11T01:13:50.686087+0000","last_became_active":"2024-09-11T01:13:50.685743+0000","last_became_peered":"2024-09-11T01:13:50.685743+0000","last_unstale":"2024-09-11T01:13:50.686087+0000","last_undegraded":"2024-09-11T01:13:50.686087+0000","last_fullsized":"2024-09-11T01:13:50.686087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:15.284517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.684090+0000","last_change":"2024-09-11T01:13:50.684001+0000","last_active":"2024-09-11T01:13:50.684090+0000","last_peered":"2024-09-11T01:13:50.684090+0000","last_clean":"2024-09-11T01:13:50.684090+0000","last_became_active":"2024-09-11T01:13:50.683734+0000","last_became_peered":"2024-09-11T01:13:50.683734+0000","last_unstale":"2024-09-11T01:13:50.684090+0000","last_undegraded":"2024-09-11T01:13:50.684090+0000","last_fullsized":"2024-09-11T01:13:50.684090+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:35:58.136208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.684811+0000","last_change":"2024-09-11T01:13:50.684700+0000","last_active":"2024-09-11T01:13:50.684811+0000","last_peered":"2024-09-11T01:13:50.684811+0000","last_clean":"2024-09-11T01:13:50.684811+0000","last_became_active":"2024-09-11T01:13:50.684354+0000","last_became_peered":"2024-09-11T01:13:50.684354+0000","last_unstale":"2024-09-11T01:13:50.684811+0000","last_undegraded":"2024-09-11T01:13:50.684811+0000","last_fullsized":"2024-09-11T01:13:50.684811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:25.882941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681464+0000","last_change":"2024-09-11T01:13:46.602152+0000","last_active":"2024-09-11T01:13:50.681464+0000","last_peered":"2024-09-11T01:13:50.681464+0000","last_clean":"2024-09-11T01:13:50.681464+0000","last_became_active":"2024-09-11T01:13:46.601872+0000","last_became_peered":"2024-09-11T01:13:46.601872+0000","last_unstale":"2024-09-11T01:13:50.681464+0000","last_undegraded":"2024-09-11T01:13:50.681464+0000","last_fullsized":"2024-09-11T01:13:50.681464+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:08.777062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681589+0000","last_change":"2024-09-11T01:13:46.602657+0000","last_active":"2024-09-11T01:13:50.681589+0000","last_peered":"2024-09-11T01:13:50.681589+0000","last_clean":"2024-09-11T01:13:50.681589+0000","last_became_active":"2024-09-11T01:13:46.600584+0000","last_became_peered":"2024-09-11T01:13:46.600584+0000","last_unstale":"2024-09-11T01:13:50.681589+0000","last_undegraded":"2024-09-11T01:13:50.681589+0000","last_fullsized":"2024-09-11T01:13:50.681589+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:17.638915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.683870+0000","last_change":"2024-09-11T01:13:50.683772+0000","last_active":"2024-09-11T01:13:50.683870+0000","last_peered":"2024-09-11T01:13:50.683870+0000","last_clean":"2024-09-11T01:13:50.683870+0000","last_became_active":"2024-09-11T01:13:50.683465+0000","last_became_peered":"2024-09-11T01:13:50.683465+0000","last_unstale":"2024-09-11T01:13:50.683870+0000","last_undegraded":"2024-09-11T01:13:50.683870+0000","last_fullsized":"2024-09-11T01:13:50.683870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:14.990045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681314+0000","last_change":"2024-09-11T01:13:38.533542+0000","last_active":"2024-09-11T01:13:50.681314+0000","last_peered":"2024-09-11T01:13:50.681314+0000","last_clean":"2024-09-11T01:13:50.681314+0000","last_became_active":"2024-09-11T01:13:36.495430+0000","last_became_peered":"2024-09-11T01:13:36.495430+0000","last_unstale":"2024-09-11T01:13:50.681314+0000","last_undegraded":"2024-09-11T01:13:50.681314+0000","last_fullsized":"2024-09-11T01:13:50.681314+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:55:23.610672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.686110+0000","last_change":"2024-09-11T01:13:50.685691+0000","last_active":"2024-09-11T01:13:50.686110+0000","last_peered":"2024-09-11T01:13:50.686110+0000","last_clean":"2024-09-11T01:13:50.686110+0000","last_became_active":"2024-09-11T01:13:50.685332+0000","last_became_peered":"2024-09-11T01:13:50.685332+0000","last_unstale":"2024-09-11T01:13:50.686110+0000","last_undegraded":"2024-09-11T01:13:50.686110+0000","last_fullsized":"2024-09-11T01:13:50.686110+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:23.690298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.684718+0000","last_change":"2024-09-11T01:13:50.684618+0000","last_active":"2024-09-11T01:13:50.684718+0000","last_peered":"2024-09-11T01:13:50.684718+0000","last_clean":"2024-09-11T01:13:50.684718+0000","last_became_active":"2024-09-11T01:13:50.684343+0000","last_became_peered":"2024-09-11T01:13:50.684343+0000","last_unstale":"2024-09-11T01:13:50.684718+0000","last_undegraded":"2024-09-11T01:13:50.684718+0000","last_fullsized":"2024-09-11T01:13:50.684718+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:42.776755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.679429+0000","last_change":"2024-09-11T01:13:46.599819+0000","last_active":"2024-09-11T01:13:50.679429+0000","last_peered":"2024-09-11T01:13:50.679429+0000","last_clean":"2024-09-11T01:13:50.679429+0000","last_became_active":"2024-09-11T01:13:46.599526+0000","last_became_peered":"2024-09-11T01:13:46.599526+0000","last_unstale":"2024-09-11T01:13:50.679429+0000","last_undegraded":"2024-09-11T01:13:50.679429+0000","last_fullsized":"2024-09-11T01:13:50.679429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:35.784033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":35,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.679500+0000","last_change":"2024-09-11T01:13:38.532991+0000","last_active":"2024-09-11T01:13:50.679500+0000","last_peered":"2024-09-11T01:13:50.679500+0000","last_clean":"2024-09-11T01:13:50.679500+0000","last_became_active":"2024-09-11T01:13:36.494051+0000","last_became_peered":"2024-09-11T01:13:36.494051+0000","last_unstale":"2024-09-11T01:13:50.679500+0000","last_undegraded":"2024-09-11T01:13:50.679500+0000","last_fullsized":"2024-09-11T01:13:50.679500+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:45:38.945658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.684927+0000","last_change":"2024-09-11T01:13:50.684833+0000","last_active":"2024-09-11T01:13:50.684927+0000","last_peered":"2024-09-11T01:13:50.684927+0000","last_clean":"2024-09-11T01:13:50.684927+0000","last_became_active":"2024-09-11T01:13:50.684573+0000","last_became_peered":"2024-09-11T01:13:50.684573+0000","last_unstale":"2024-09-11T01:13:50.684927+0000","last_undegraded":"2024-09-11T01:13:50.684927+0000","last_fullsized":"2024-09-11T01:13:50.684927+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:15.073754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.690530+0000","last_change":"2024-09-11T01:13:50.690440+0000","last_active":"2024-09-11T01:13:50.690530+0000","last_peered":"2024-09-11T01:13:50.690530+0000","last_clean":"2024-09-11T01:13:50.690530+0000","last_became_active":"2024-09-11T01:13:50.685884+0000","last_became_peered":"2024-09-11T01:13:50.685884+0000","last_unstale":"2024-09-11T01:13:50.690530+0000","last_undegraded":"2024-09-11T01:13:50.690530+0000","last_fullsized":"2024-09-11T01:13:50.690530+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:13.955930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.685652+0000","last_change":"2024-09-11T01:13:50.685531+0000","last_active":"2024-09-11T01:13:50.685652+0000","last_peered":"2024-09-11T01:13:50.685652+0000","last_clean":"2024-09-11T01:13:50.685652+0000","last_became_active":"2024-09-11T01:13:50.685206+0000","last_became_peered":"2024-09-11T01:13:50.685206+0000","last_unstale":"2024-09-11T01:13:50.685652+0000","last_undegraded":"2024-09-11T01:13:50.685652+0000","last_fullsized":"2024-09-11T01:13:50.685652+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:09.662455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681885+0000","last_change":"2024-09-11T01:13:46.602073+0000","last_active":"2024-09-11T01:13:50.681885+0000","last_peered":"2024-09-11T01:13:50.681885+0000","last_clean":"2024-09-11T01:13:50.681885+0000","last_became_active":"2024-09-11T01:13:46.601706+0000","last_became_peered":"2024-09-11T01:13:46.601706+0000","last_unstale":"2024-09-11T01:13:50.681885+0000","last_undegraded":"2024-09-11T01:13:50.681885+0000","last_fullsized":"2024-09-11T01:13:50.681885+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:31:15.458280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681686+0000","last_change":"2024-09-11T01:13:38.533379+0000","last_active":"2024-09-11T01:13:50.681686+0000","last_peered":"2024-09-11T01:13:50.681686+0000","last_clean":"2024-09-11T01:13:50.681686+0000","last_became_active":"2024-09-11T01:13:36.494318+0000","last_became_peered":"2024-09-11T01:13:36.494318+0000","last_unstale":"2024-09-11T01:13:50.681686+0000","last_undegraded":"2024-09-11T01:13:50.681686+0000","last_fullsized":"2024-09-11T01:13:50.681686+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:25.161001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681730+0000","last_change":"2024-09-11T01:13:46.602379+0000","last_active":"2024-09-11T01:13:50.681730+0000","last_peered":"2024-09-11T01:13:50.681730+0000","last_clean":"2024-09-11T01:13:50.681730+0000","last_became_active":"2024-09-11T01:13:46.601931+0000","last_became_peered":"2024-09-11T01:13:46.601931+0000","last_unstale":"2024-09-11T01:13:50.681730+0000","last_undegraded":"2024-09-11T01:13:50.681730+0000","last_fullsized":"2024-09-11T01:13:50.681730+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:25.833664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.684716+0000","last_change":"2024-09-11T01:13:50.684613+0000","last_active":"2024-09-11T01:13:50.684716+0000","last_peered":"2024-09-11T01:13:50.684716+0000","last_clean":"2024-09-11T01:13:50.684716+0000","last_became_active":"2024-09-11T01:13:50.684286+0000","last_became_peered":"2024-09-11T01:13:50.684286+0000","last_unstale":"2024-09-11T01:13:50.684716+0000","last_undegraded":"2024-09-11T01:13:50.684716+0000","last_fullsized":"2024-09-11T01:13:50.684716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:41:53.876024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681792+0000","last_change":"2024-09-11T01:13:46.600876+0000","last_active":"2024-09-11T01:13:50.681792+0000","last_peered":"2024-09-11T01:13:50.681792+0000","last_clean":"2024-09-11T01:13:50.681792+0000","last_became_active":"2024-09-11T01:13:46.600623+0000","last_became_peered":"2024-09-11T01:13:46.600623+0000","last_unstale":"2024-09-11T01:13:50.681792+0000","last_undegraded":"2024-09-11T01:13:50.681792+0000","last_fullsized":"2024-09-11T01:13:50.681792+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:10.864417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681866+0000","last_change":"2024-09-11T01:13:38.534017+0000","last_active":"2024-09-11T01:13:50.681866+0000","last_peered":"2024-09-11T01:13:50.681866+0000","last_clean":"2024-09-11T01:13:50.681866+0000","last_became_active":"2024-09-11T01:13:36.494215+0000","last_became_peered":"2024-09-11T01:13:36.494215+0000","last_unstale":"2024-09-11T01:13:50.681866+0000","last_undegraded":"2024-09-11T01:13:50.681866+0000","last_fullsized":"2024-09-11T01:13:50.681866+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:51:50.232692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.686222+0000","last_change":"2024-09-11T01:13:50.686129+0000","last_active":"2024-09-11T01:13:50.686222+0000","last_peered":"2024-09-11T01:13:50.686222+0000","last_clean":"2024-09-11T01:13:50.686222+0000","last_became_active":"2024-09-11T01:13:50.685823+0000","last_became_peered":"2024-09-11T01:13:50.685823+0000","last_unstale":"2024-09-11T01:13:50.686222+0000","last_undegraded":"2024-09-11T01:13:50.686222+0000","last_fullsized":"2024-09-11T01:13:50.686222+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:40.702954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:13:50.681657+0000","last_change":"2024-09-11T01:13:46.600813+0000","last_active":"2024-09-11T01:13:50.681657+0000","last_peered":"2024-09-11T01:13:50.681657+0000","last_clean":"2024-09-11T01:13:50.681657+0000","last_became_active":"2024-09-11T01:13:46.600533+0000","last_became_peered":"2024-09-11T01:13:46.600533+0000","last_unstale":"2024-09-11T01:13:50.681657+0000","last_undegraded":"2024-09-11T01:13:50.681657+0000","last_fullsized":"2024-09-11T01:13:50.681657+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:39.388587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":188416,"data_stored":49779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738377,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":188416,"data_stored":49779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:59.515 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-11T01:13:59.515 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-11T01:13:59.516 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-11T01:13:59.568 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-11T01:13:59.568 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-11T01:13:59.621 INFO:tasks.util.rados:rados -p unfound_pool rm existed_1 2024-09-11T01:13:59.621 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_1 2024-09-11T01:13:59.672 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-11T01:13:59.672 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-11T01:13:59.730 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-11T01:13:59.730 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-11T01:13:59.782 INFO:tasks.util.rados:rados -p unfound_pool rm existed_2 2024-09-11T01:13:59.782 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_2 2024-09-11T01:13:59.833 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-11T01:13:59.833 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-11T01:13:59.885 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-11T01:13:59.885 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-11T01:13:59.936 INFO:tasks.util.rados:rados -p unfound_pool rm existed_3 2024-09-11T01:13:59.937 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_3 2024-09-11T01:13:59.987 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-11T01:13:59.987 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-11T01:14:00.039 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-11T01:14:00.039 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-11T01:14:00.091 INFO:tasks.util.rados:rados -p unfound_pool rm existed_4 2024-09-11T01:14:00.091 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_4 2024-09-11T01:14:00.141 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-11T01:14:00.142 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-11T01:14:00.193 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-11T01:14:00.194 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-11T01:14:00.245 INFO:tasks.util.rados:rados -p unfound_pool rm existed_5 2024-09-11T01:14:00.245 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_5 2024-09-11T01:14:00.296 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-11T01:14:00.296 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-11T01:14:00.349 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-11T01:14:00.349 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-11T01:14:00.401 INFO:tasks.util.rados:rados -p unfound_pool rm existed_6 2024-09-11T01:14:00.401 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_6 2024-09-11T01:14:00.452 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-11T01:14:00.452 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-11T01:14:00.505 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-11T01:14:00.505 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-11T01:14:00.558 INFO:tasks.util.rados:rados -p unfound_pool rm existed_7 2024-09-11T01:14:00.558 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_7 2024-09-11T01:14:00.609 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-11T01:14:00.609 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-11T01:14:00.660 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-11T01:14:00.661 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-11T01:14:00.714 INFO:tasks.util.rados:rados -p unfound_pool rm existed_8 2024-09-11T01:14:00.715 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_8 2024-09-11T01:14:00.765 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-11T01:14:00.766 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-11T01:14:00.817 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-11T01:14:00.818 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-11T01:14:00.869 INFO:tasks.util.rados:rados -p unfound_pool rm existed_9 2024-09-11T01:14:00.869 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_9 2024-09-11T01:14:00.920 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-11T01:14:01.044 INFO:teuthology.orchestra.run.smithi039.stdout:{} 2024-09-11T01:14:01.044 INFO:teuthology.orchestra.run.smithi039.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-11T01:14:01.055 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T01:14:01.055 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:14:01.085 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T01:14:01.085 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-11T01:14:01.432 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:01.430+0000 7f7c4de44640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-11T01:14:01.730 INFO:teuthology.orchestra.run.smithi039.stderr:marked down osd.0. 2024-09-11T01:14:01.745 INFO:tasks.util.rados:rados -p unfound_pool put new_1 /etc/fstab 2024-09-11T01:14:01.745 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_1 /etc/fstab 2024-09-11T01:14:02.757 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-11T01:14:02.758 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-11T01:14:02.815 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-11T01:14:02.815 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-11T01:14:02.869 INFO:tasks.util.rados:rados -p unfound_pool put new_2 /etc/fstab 2024-09-11T01:14:02.869 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_2 /etc/fstab 2024-09-11T01:14:02.921 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-11T01:14:02.922 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-11T01:14:02.973 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-11T01:14:02.974 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-11T01:14:03.025 INFO:tasks.util.rados:rados -p unfound_pool put new_3 /etc/fstab 2024-09-11T01:14:03.025 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_3 /etc/fstab 2024-09-11T01:14:03.077 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-11T01:14:03.077 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-11T01:14:03.128 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-11T01:14:03.128 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-11T01:14:03.180 INFO:tasks.util.rados:rados -p unfound_pool put new_4 /etc/fstab 2024-09-11T01:14:03.180 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_4 /etc/fstab 2024-09-11T01:14:03.433 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-11T01:14:03.433 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-11T01:14:03.485 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-11T01:14:03.485 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-11T01:14:03.537 INFO:tasks.util.rados:rados -p unfound_pool put new_5 /etc/fstab 2024-09-11T01:14:03.537 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_5 /etc/fstab 2024-09-11T01:14:03.589 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-11T01:14:03.590 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-11T01:14:03.641 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-11T01:14:03.641 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-11T01:14:03.693 INFO:tasks.util.rados:rados -p unfound_pool put new_6 /etc/fstab 2024-09-11T01:14:03.693 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_6 /etc/fstab 2024-09-11T01:14:03.744 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-11T01:14:03.745 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-11T01:14:03.796 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-11T01:14:03.797 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-11T01:14:03.848 INFO:tasks.util.rados:rados -p unfound_pool put new_7 /etc/fstab 2024-09-11T01:14:03.848 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_7 /etc/fstab 2024-09-11T01:14:03.900 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-11T01:14:03.900 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-11T01:14:03.952 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-11T01:14:03.952 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-11T01:14:04.004 INFO:tasks.util.rados:rados -p unfound_pool put new_8 /etc/fstab 2024-09-11T01:14:04.004 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_8 /etc/fstab 2024-09-11T01:14:04.055 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-11T01:14:04.055 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-11T01:14:04.107 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-11T01:14:04.107 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-11T01:14:04.158 INFO:tasks.util.rados:rados -p unfound_pool put new_9 /etc/fstab 2024-09-11T01:14:04.158 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_9 /etc/fstab 2024-09-11T01:14:04.209 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-11T01:14:04.210 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-11T01:14:04.262 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-11T01:14:04.262 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-11T01:14:04.314 INFO:tasks.lost_unfound:osd.0 command_args is foo 2024-09-11T01:14:04.314 INFO:tasks.lost_unfound:() 2024-09-11T01:14:04.314 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T01:14:04.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-11T01:14:04.316 INFO:tasks.ceph.osd.0:Started 2024-09-11T01:14:04.320 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T01:14:04.422 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-11T01:14:04.422 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-11T01:14:04.422 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-11T01:14:04.422 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:config diff 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:config diff get 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:config get 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:config help [] 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:config set ... 2024-09-11T01:14:04.423 INFO:teuthology.orchestra.run.smithi039.stderr:admin_socket: invalid command 2024-09-11T01:14:04.424 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:14:04.425 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-11T01:14:05.247 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:05.246+0000 7f06a31b0640 -1 Falling back to public interface 2024-09-11T01:14:09.086 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:09.085+0000 7f06a31b0640 -1 osd.0 21 log_to_monitors true 2024-09-11T01:14:09.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T01:14:09.531 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator fragmentation block 2024-09-11T01:14:09.532 INFO:teuthology.orchestra.run.smithi039.stderr:admin_socket: invalid command 2024-09-11T01:14:09.534 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:14:09.534 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-11T01:14:09.828 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:09.826+0000 7f069af66640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:14:14.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T01:14:14.656 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:14.656 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-11T01:14:14.656 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-11T01:14:14.656 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:14.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-11T01:14:14.976 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:14.975+0000 7f7c4de44640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-11T01:14:14.977 INFO:teuthology.orchestra.run.smithi039.stderr:osd.0 is already in. 2024-09-11T01:14:14.990 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.0 to be up 2024-09-11T01:14:14.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:14:15.286 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:15.286 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":26,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:14:11.822152+0000","last_up_change":"2024-09-11T01:14:10.818140+0000","last_in_change":"2024-09-11T01:13:49.646901+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-11T01:13:44.644516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2566795661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":2566795661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":2566795661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":2566795661}]},"public_addr":"172.21.15.39:6800/2566795661","cluster_addr":"172.21.15.39:6801/2566795661","heartbeat_back_addr":"172.21.15.39:6803/2566795661","heartbeat_front_addr":"172.21.15.39:6802/2566795661","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3162293523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":3162293523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":3162293523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3162293523}]},"public_addr":"172.21.15.39:6804/3162293523","cluster_addr":"172.21.15.39:6805/3162293523","heartbeat_back_addr":"172.21.15.39:6807/3162293523","heartbeat_front_addr":"172.21.15.39:6806/3162293523","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":3298933028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":3298933028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":3298933028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":3298933028}]},"public_addr":"172.21.15.39:6808/3298933028","cluster_addr":"172.21.15.39:6809/3298933028","heartbeat_back_addr":"172.21.15.39:6811/3298933028","heartbeat_front_addr":"172.21.15.39:6810/3298933028","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:14:15.298 INFO:tasks.lost_unfound.ceph_manager:osd.0 is up 2024-09-11T01:14:15.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:14:15.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:14:15.435 INFO:teuthology.orchestra.run.smithi039.stdout:107374182403 2024-09-11T01:14:15.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:15.437 INFO:teuthology.orchestra.run.smithi039.stdout:38654705678 2024-09-11T01:14:15.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:14:15.777 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-11T01:14:15.784 INFO:teuthology.orchestra.run.smithi039.stdout:38654705676 2024-09-11T01:14:15.792 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-11T01:14:15.798 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705678 got 38654705676 for osd.1 2024-09-11T01:14:16.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:16.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:14:17.113 INFO:teuthology.orchestra.run.smithi039.stdout:107374182403 2024-09-11T01:14:17.126 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 107374182403 for osd.0 2024-09-11T01:14:17.126 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:17.146 INFO:teuthology.orchestra.run.smithi039.stdout:38654705678 2024-09-11T01:14:17.159 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705678 got 38654705678 for osd.1 2024-09-11T01:14:17.159 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:17.159 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-11T01:14:17.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:17.492 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:17.492 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:17.504 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-11T01:14:16.404202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":59264,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":58618,"kb_avail":188684416,"statfs":{"total":193273528320,"available":193212841984,"internally_reserved":0,"allocated":606208,"data_stored":147884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6033,"internal_metadata":60024943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.583154"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.842160+0000","last_change":"2024-09-11T01:14:11.842060+0000","last_active":"2024-09-11T01:14:11.842160+0000","last_peered":"2024-09-11T01:14:11.842160+0000","last_clean":"2024-09-11T01:14:11.842160+0000","last_became_active":"2024-09-11T01:14:11.841641+0000","last_became_peered":"2024-09-11T01:14:11.841641+0000","last_unstale":"2024-09-11T01:14:11.842160+0000","last_undegraded":"2024-09-11T01:14:11.842160+0000","last_fullsized":"2024-09-11T01:14:11.842160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:35:41.989927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":24,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.841932+0000","last_change":"2024-09-11T01:14:11.842346+0000","last_active":"2024-09-11T01:14:11.841932+0000","last_peered":"2024-09-11T01:14:11.841932+0000","last_clean":"2024-09-11T01:14:00.035698+0000","last_became_active":"2024-09-11T01:14:11.841800+0000","last_became_peered":"2024-09-11T01:14:11.841800+0000","last_unstale":"2024-09-11T01:14:11.841932+0000","last_undegraded":"2024-09-11T01:14:11.831472+0000","last_fullsized":"2024-09-11T01:14:11.841932+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":44,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.842412+0000","last_change":"2024-09-11T01:14:11.842599+0000","last_active":"2024-09-11T01:14:11.842412+0000","last_peered":"2024-09-11T01:14:11.842412+0000","last_clean":"2024-09-11T01:14:00.605389+0000","last_became_active":"2024-09-11T01:14:11.842317+0000","last_became_peered":"2024-09-11T01:14:11.842317+0000","last_unstale":"2024-09-11T01:14:11.842412+0000","last_undegraded":"2024-09-11T01:14:11.831193+0000","last_fullsized":"2024-09-11T01:14:11.842412+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.842729+0000","last_change":"2024-09-11T01:14:11.842564+0000","last_active":"2024-09-11T01:14:11.842729+0000","last_peered":"2024-09-11T01:14:11.842729+0000","last_clean":"2024-09-11T01:14:11.842729+0000","last_became_active":"2024-09-11T01:14:11.842224+0000","last_became_peered":"2024-09-11T01:14:11.842224+0000","last_unstale":"2024-09-11T01:14:11.842729+0000","last_undegraded":"2024-09-11T01:14:11.842729+0000","last_fullsized":"2024-09-11T01:14:11.842729+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:23.912190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.842901+0000","last_change":"2024-09-11T01:14:11.843091+0000","last_active":"2024-09-11T01:14:11.842901+0000","last_peered":"2024-09-11T01:14:11.842901+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:11.842435+0000","last_became_peered":"2024-09-11T01:14:11.842435+0000","last_unstale":"2024-09-11T01:14:11.842901+0000","last_undegraded":"2024-09-11T01:14:11.831128+0000","last_fullsized":"2024-09-11T01:14:11.842901+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.835817+0000","last_change":"2024-09-11T01:14:11.835997+0000","last_active":"2024-09-11T01:14:11.835817+0000","last_peered":"2024-09-11T01:14:11.835817+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:11.835703+0000","last_became_peered":"2024-09-11T01:14:11.835703+0000","last_unstale":"2024-09-11T01:14:11.835817+0000","last_undegraded":"2024-09-11T01:14:11.830888+0000","last_fullsized":"2024-09-11T01:14:11.835817+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.839884+0000","last_change":"2024-09-11T01:14:11.840090+0000","last_active":"2024-09-11T01:14:11.839884+0000","last_peered":"2024-09-11T01:14:11.839884+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:11.839755+0000","last_became_peered":"2024-09-11T01:14:11.839755+0000","last_unstale":"2024-09-11T01:14:11.839884+0000","last_undegraded":"2024-09-11T01:14:11.830909+0000","last_fullsized":"2024-09-11T01:14:11.839884+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":31,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.843011+0000","last_change":"2024-09-11T01:14:11.843191+0000","last_active":"2024-09-11T01:14:11.843011+0000","last_peered":"2024-09-11T01:14:11.843011+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:11.842923+0000","last_became_peered":"2024-09-11T01:14:11.842923+0000","last_unstale":"2024-09-11T01:14:11.843011+0000","last_undegraded":"2024-09-11T01:14:11.829172+0000","last_fullsized":"2024-09-11T01:14:11.843011+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.840567+0000","last_change":"2024-09-11T01:14:11.840404+0000","last_active":"2024-09-11T01:14:11.840567+0000","last_peered":"2024-09-11T01:14:11.840567+0000","last_clean":"2024-09-11T01:14:11.840567+0000","last_became_active":"2024-09-11T01:14:11.839569+0000","last_became_peered":"2024-09-11T01:14:11.839569+0000","last_unstale":"2024-09-11T01:14:11.840567+0000","last_undegraded":"2024-09-11T01:14:11.840567+0000","last_fullsized":"2024-09-11T01:14:11.840567+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:52.969876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":46,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.835707+0000","last_change":"2024-09-11T01:14:11.836082+0000","last_active":"2024-09-11T01:14:11.835707+0000","last_peered":"2024-09-11T01:14:11.835707+0000","last_clean":"2024-09-11T01:14:00.657239+0000","last_became_active":"2024-09-11T01:14:11.835595+0000","last_became_peered":"2024-09-11T01:14:11.835595+0000","last_unstale":"2024-09-11T01:14:11.835707+0000","last_undegraded":"2024-09-11T01:14:11.830626+0000","last_fullsized":"2024-09-11T01:14:11.835707+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.841414+0000","last_change":"2024-09-11T01:14:11.841290+0000","last_active":"2024-09-11T01:14:11.841414+0000","last_peered":"2024-09-11T01:14:11.841414+0000","last_clean":"2024-09-11T01:14:11.841414+0000","last_became_active":"2024-09-11T01:14:11.840926+0000","last_became_peered":"2024-09-11T01:14:11.840926+0000","last_unstale":"2024-09-11T01:14:11.841414+0000","last_undegraded":"2024-09-11T01:14:11.841414+0000","last_fullsized":"2024-09-11T01:14:11.841414+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:19.425081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":32,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:14:11.842128+0000","last_change":"2024-09-11T01:14:11.844396+0000","last_active":"2024-09-11T01:14:11.842128+0000","last_peered":"2024-09-11T01:14:11.842128+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:11.841790+0000","last_became_peered":"2024-09-11T01:14:11.841790+0000","last_unstale":"2024-09-11T01:14:11.842128+0000","last_undegraded":"2024-09-11T01:14:11.830399+0000","last_fullsized":"2024-09-11T01:14:11.842128+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.842104+0000","last_change":"2024-09-11T01:14:11.841996+0000","last_active":"2024-09-11T01:14:11.842104+0000","last_peered":"2024-09-11T01:14:11.842104+0000","last_clean":"2024-09-11T01:14:11.842104+0000","last_became_active":"2024-09-11T01:14:11.841485+0000","last_became_peered":"2024-09-11T01:14:11.841485+0000","last_unstale":"2024-09-11T01:14:11.842104+0000","last_undegraded":"2024-09-11T01:14:11.842104+0000","last_fullsized":"2024-09-11T01:14:11.842104+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:21.923002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":33,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.839334+0000","last_change":"2024-09-11T01:14:11.839555+0000","last_active":"2024-09-11T01:14:11.839334+0000","last_peered":"2024-09-11T01:14:11.839334+0000","last_clean":"2024-09-11T01:13:59.881324+0000","last_became_active":"2024-09-11T01:14:11.839232+0000","last_became_peered":"2024-09-11T01:14:11.839232+0000","last_unstale":"2024-09-11T01:14:11.839334+0000","last_undegraded":"2024-09-11T01:14:11.830798+0000","last_fullsized":"2024-09-11T01:14:11.839334+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.840429+0000","last_change":"2024-09-11T01:14:11.840295+0000","last_active":"2024-09-11T01:14:11.840429+0000","last_peered":"2024-09-11T01:14:11.840429+0000","last_clean":"2024-09-11T01:14:11.840429+0000","last_became_active":"2024-09-11T01:14:11.839795+0000","last_became_peered":"2024-09-11T01:14:11.839795+0000","last_unstale":"2024-09-11T01:14:11.840429+0000","last_undegraded":"2024-09-11T01:14:11.840429+0000","last_fullsized":"2024-09-11T01:14:11.840429+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:43:38.297043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.840368+0000","last_change":"2024-09-11T01:14:11.840582+0000","last_active":"2024-09-11T01:14:11.840368+0000","last_peered":"2024-09-11T01:14:11.840368+0000","last_clean":"2024-09-11T01:14:00.292751+0000","last_became_active":"2024-09-11T01:14:11.840277+0000","last_became_peered":"2024-09-11T01:14:11.840277+0000","last_unstale":"2024-09-11T01:14:11.840368+0000","last_undegraded":"2024-09-11T01:14:11.829723+0000","last_fullsized":"2024-09-11T01:14:11.840368+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.835930+0000","last_change":"2024-09-11T01:14:11.836130+0000","last_active":"2024-09-11T01:14:11.835930+0000","last_peered":"2024-09-11T01:14:11.835930+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:11.835807+0000","last_became_peered":"2024-09-11T01:14:11.835807+0000","last_unstale":"2024-09-11T01:14:11.835930+0000","last_undegraded":"2024-09-11T01:14:11.830365+0000","last_fullsized":"2024-09-11T01:14:11.835930+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.840034+0000","last_change":"2024-09-11T01:14:11.839912+0000","last_active":"2024-09-11T01:14:11.840034+0000","last_peered":"2024-09-11T01:14:11.840034+0000","last_clean":"2024-09-11T01:14:11.840034+0000","last_became_active":"2024-09-11T01:14:11.839272+0000","last_became_peered":"2024-09-11T01:14:11.839272+0000","last_unstale":"2024-09-11T01:14:11.840034+0000","last_undegraded":"2024-09-11T01:14:11.840034+0000","last_fullsized":"2024-09-11T01:14:11.840034+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:02.986379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.836178+0000","last_change":"2024-09-11T01:14:11.836370+0000","last_active":"2024-09-11T01:14:11.836178+0000","last_peered":"2024-09-11T01:14:11.836178+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:11.835650+0000","last_became_peered":"2024-09-11T01:14:11.835650+0000","last_unstale":"2024-09-11T01:14:11.836178+0000","last_undegraded":"2024-09-11T01:14:11.829870+0000","last_fullsized":"2024-09-11T01:14:11.836178+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.842344+0000","last_change":"2024-09-11T01:14:11.842025+0000","last_active":"2024-09-11T01:14:11.842344+0000","last_peered":"2024-09-11T01:14:11.842344+0000","last_clean":"2024-09-11T01:14:11.842344+0000","last_became_active":"2024-09-11T01:14:11.841533+0000","last_became_peered":"2024-09-11T01:14:11.841533+0000","last_unstale":"2024-09-11T01:14:11.842344+0000","last_undegraded":"2024-09-11T01:14:11.842344+0000","last_fullsized":"2024-09-11T01:14:11.842344+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:46.835814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":37,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:11.842535+0000","last_change":"2024-09-11T01:14:11.842733+0000","last_active":"2024-09-11T01:14:11.842535+0000","last_peered":"2024-09-11T01:14:11.842535+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:11.842427+0000","last_became_peered":"2024-09-11T01:14:11.842427+0000","last_unstale":"2024-09-11T01:14:11.842535+0000","last_undegraded":"2024-09-11T01:14:11.831454+0000","last_fullsized":"2024-09-11T01:14:11.842535+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.839647+0000","last_change":"2024-09-11T01:14:11.839538+0000","last_active":"2024-09-11T01:14:11.839647+0000","last_peered":"2024-09-11T01:14:11.839647+0000","last_clean":"2024-09-11T01:14:11.839647+0000","last_became_active":"2024-09-11T01:14:11.839198+0000","last_became_peered":"2024-09-11T01:14:11.839198+0000","last_unstale":"2024-09-11T01:14:11.839647+0000","last_undegraded":"2024-09-11T01:14:11.839647+0000","last_fullsized":"2024-09-11T01:14:11.839647+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:20.478132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:14:11.836283+0000","last_change":"2024-09-11T01:14:11.836169+0000","last_active":"2024-09-11T01:14:11.836283+0000","last_peered":"2024-09-11T01:14:11.836283+0000","last_clean":"2024-09-11T01:14:11.836283+0000","last_became_active":"2024-09-11T01:14:11.835748+0000","last_became_peered":"2024-09-11T01:14:11.835748+0000","last_unstale":"2024-09-11T01:14:11.836283+0000","last_undegraded":"2024-09-11T01:14:11.836283+0000","last_fullsized":"2024-09-11T01:14:11.836283+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:21.951993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":46,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:14:11.835961+0000","last_change":"2024-09-11T01:14:11.836904+0000","last_active":"2024-09-11T01:14:11.835961+0000","last_peered":"2024-09-11T01:14:11.835961+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:11.835592+0000","last_became_peered":"2024-09-11T01:14:11.835592+0000","last_unstale":"2024-09-11T01:14:11.835961+0000","last_undegraded":"2024-09-11T01:14:11.829808+0000","last_fullsized":"2024-09-11T01:14:11.835961+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705678,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":78289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":266240,"data_stored":69595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":32370346},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:17.506 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:17.871 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:17.872 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":58,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017250,"num_in_osds":2,"osd_in_since":1726017229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":60686336,"bytes_avail":193212841984,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-11T01:13:19.553908+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:17.884 INFO:tasks.lost_unfound.ceph_manager:{'fsid': 'bcb624ec-d02e-4f87-a117-bb3a3b752391', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 4, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 58, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017250, 'num_in_osds': 2, 'osd_in_since': 1726017229, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 60686336, 'bytes_avail': 193212841984, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:13:19:552800+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-11T01:13:19.553908+0000', 'services': {}}, 'progress_events': {}} 2024-09-11T01:14:17.884 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-11T01:14:17.885 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T01:14:17.885 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:14:17.908 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T01:14:17.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-11T01:14:18.257 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:18.256+0000 7f7c4de44640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-11T01:14:18.258 INFO:teuthology.orchestra.run.smithi039.stderr:osd.1 is already down. 2024-09-11T01:14:18.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-11T01:14:18.578 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:18.577+0000 7f7c4de44640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-11T01:14:18.927 INFO:teuthology.orchestra.run.smithi039.stderr:marked out osd.1. 2024-09-11T01:14:18.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-11T01:14:19.957 INFO:teuthology.orchestra.run.smithi039.stderr:marked osd lost in epoch 27 2024-09-11T01:14:19.970 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.966+0000 7f0685611640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T01:14:19.970 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0689619640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T01:14:19.970 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0684e10640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T01:14:19.970 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f068260b640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0688e18640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0686e14640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0688617640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0682e0c640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0687615640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0688617640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0686613640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.967+0000 7f0685e12640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.968+0000 7f0687e16640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T01:14:19.971 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:19.968+0000 7f0683e0e640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T01:14:19.976 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T01:14:19.977 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T01:14:19.979 INFO:tasks.ceph.osd.2:Started 2024-09-11T01:14:19.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:config diff 2024-09-11T01:14:20.086 INFO:teuthology.orchestra.run.smithi039.stderr:config diff get 2024-09-11T01:14:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:config get 2024-09-11T01:14:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:config help [] 2024-09-11T01:14:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:config set ... 2024-09-11T01:14:20.087 INFO:teuthology.orchestra.run.smithi039.stderr:admin_socket: invalid command 2024-09-11T01:14:20.088 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:14:20.089 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-11T01:14:20.894 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:14:20.892+0000 7f912fc14640 -1 Falling back to public interface 2024-09-11T01:14:24.406 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:14:24.404+0000 7f912fc14640 -1 osd.2 17 log_to_monitors true 2024-09-11T01:14:24.464 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.462+0000 7f0689619640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T01:14:24.464 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.462+0000 7f0684e10640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T01:14:24.464 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.462+0000 7f0685611640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T01:14:24.465 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.462+0000 7f0684e10640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:14:24.465 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f068460f640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0682e0c640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0688617640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0682e0c640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0683e0e640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f068260b640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0687e16640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0685e12640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f0687615640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:14:24.466 INFO:tasks.ceph.osd.0.smithi039.stderr:2024-09-11T01:14:24.463+0000 7f068260b640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:14:25.000 INFO:tasks.ceph.osd.2.smithi039.stderr:2024-09-11T01:14:24.998+0000 7f9123399640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:14:25.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T01:14:25.197 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:25.197 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [], 2024-09-11T01:14:25.197 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 0 2024-09-11T01:14:25.197 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:25.206 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-11T01:14:25.516 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:25.515+0000 7f7c4de44640 -1 mon.a@0(leader).osd e30 definitely_dead 0 2024-09-11T01:14:26.013 INFO:teuthology.orchestra.run.smithi039.stderr:marked in osd.2. 2024-09-11T01:14:26.031 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.2 to be up 2024-09-11T01:14:26.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:14:26.330 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:26.330 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":31,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:14:26.000314+0000","last_up_change":"2024-09-11T01:14:26.000314+0000","last_in_change":"2024-09-11T01:14:26.000314+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-11T01:13:44.644516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":27,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2566795661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":2566795661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":2566795661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":2566795661}]},"public_addr":"172.21.15.39:6800/2566795661","cluster_addr":"172.21.15.39:6801/2566795661","heartbeat_back_addr":"172.21.15.39:6803/2566795661","heartbeat_front_addr":"172.21.15.39:6802/2566795661","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":3162293523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":3162293523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":3162293523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":3162293523}]},"public_addr":"172.21.15.39:6804/3162293523","cluster_addr":"172.21.15.39:6805/3162293523","heartbeat_back_addr":"172.21.15.39:6807/3162293523","heartbeat_front_addr":"172.21.15.39:6806/3162293523","state":["exists"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":833967465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":833967465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":833967465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":833967465}]},"public_addr":"172.21.15.39:6804/833967465","cluster_addr":"172.21.15.39:6805/833967465","heartbeat_back_addr":"172.21.15.39:6807/833967465","heartbeat_front_addr":"172.21.15.39:6806/833967465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:14:26.343 INFO:tasks.lost_unfound.ceph_manager:osd.2 is up 2024-09-11T01:14:26.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:14:26.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:14:26.482 INFO:teuthology.orchestra.run.smithi039.stdout:133143986178 2024-09-11T01:14:26.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:26.484 INFO:teuthology.orchestra.run.smithi039.stdout:107374182406 2024-09-11T01:14:26.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:26.826 INFO:teuthology.orchestra.run.smithi039.stdout:0 2024-09-11T01:14:26.827 INFO:teuthology.orchestra.run.smithi039.stdout:107374182404 2024-09-11T01:14:26.839 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986178 got 0 for osd.2 2024-09-11T01:14:26.841 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182406 got 107374182404 for osd.0 2024-09-11T01:14:27.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:27.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:28.156 INFO:teuthology.orchestra.run.smithi039.stdout:107374182405 2024-09-11T01:14:28.162 INFO:teuthology.orchestra.run.smithi039.stdout:1 2024-09-11T01:14:28.170 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182406 got 107374182405 for osd.0 2024-09-11T01:14:28.175 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986178 got 1 for osd.2 2024-09-11T01:14:29.171 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:29.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:29.503 INFO:teuthology.orchestra.run.smithi039.stdout:107374182406 2024-09-11T01:14:29.516 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182406 got 107374182406 for osd.0 2024-09-11T01:14:29.516 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:29.529 INFO:teuthology.orchestra.run.smithi039.stdout:133143986178 2024-09-11T01:14:29.541 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986178 got 133143986178 for osd.2 2024-09-11T01:14:29.542 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:29.542 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-11T01:14:29.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:29.878 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:29.878 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:29.891 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2024-09-11T01:14:28.406349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":63888,"kb_used_data":548,"kb_used_omap":8,"kb_used_meta":63287,"kb_avail":188679792,"statfs":{"total":193273528320,"available":193208107008,"internally_reserved":0,"allocated":561152,"data_stored":161947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":64806232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001160"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":34,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.464123+0000","last_change":"2024-09-11T01:14:26.012021+0000","last_active":"2024-09-11T01:14:26.012021+0000","last_peered":"2024-09-11T01:14:24.464123+0000","last_clean":"2024-09-11T01:14:11.846565+0000","last_became_active":"2024-09-11T01:14:18.943435+0000","last_became_peered":"2024-09-11T01:14:18.943435+0000","last_unstale":"2024-09-11T01:14:24.464123+0000","last_undegraded":"2024-09-11T01:14:24.464123+0000","last_fullsized":"2024-09-11T01:14:18.938757+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":13,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:24.463996+0000","last_change":"2024-09-11T01:14:26.021973+0000","last_active":"2024-09-11T01:14:26.021973+0000","last_peered":"2024-09-11T01:14:24.463996+0000","last_clean":"2024-09-11T01:14:00.035698+0000","last_became_active":"2024-09-11T01:14:18.941434+0000","last_became_peered":"2024-09-11T01:14:18.941434+0000","last_unstale":"2024-09-11T01:14:24.463996+0000","last_undegraded":"2024-09-11T01:14:18.937588+0000","last_fullsized":"2024-09-11T01:14:18.937470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":6,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:26.022287+0000","last_change":"2024-09-11T01:14:26.022287+0000","last_active":"2024-09-11T01:14:26.022287+0000","last_peered":"2024-09-11T01:14:24.463483+0000","last_clean":"2024-09-11T01:14:00.605389+0000","last_became_active":"2024-09-11T01:14:18.941510+0000","last_became_peered":"2024-09-11T01:14:18.941510+0000","last_unstale":"2024-09-11T01:14:26.022287+0000","last_undegraded":"2024-09-11T01:14:26.022287+0000","last_fullsized":"2024-09-11T01:14:26.022287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:26.014687+0000","last_change":"2024-09-11T01:14:26.013797+0000","last_active":"2024-09-11T01:14:24.470884+0000","last_peered":"2024-09-11T01:13:46.609490+0000","last_clean":"2024-09-11T01:13:46.609490+0000","last_became_active":"2024-09-11T01:13:46.609028+0000","last_became_peered":"2024-09-11T01:13:46.609028+0000","last_unstale":"2024-09-11T01:14:26.014687+0000","last_undegraded":"2024-09-11T01:14:26.014687+0000","last_fullsized":"2024-09-11T01:14:26.014687+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":45,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.464062+0000","last_change":"2024-09-11T01:14:26.011018+0000","last_active":"2024-09-11T01:14:26.011018+0000","last_peered":"2024-09-11T01:14:24.464062+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:18.943276+0000","last_became_peered":"2024-09-11T01:14:18.943276+0000","last_unstale":"2024-09-11T01:14:24.464062+0000","last_undegraded":"2024-09-11T01:14:18.939129+0000","last_fullsized":"2024-09-11T01:14:18.939010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":46,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463593+0000","last_change":"2024-09-11T01:14:26.010996+0000","last_active":"2024-09-11T01:14:26.010996+0000","last_peered":"2024-09-11T01:14:24.463593+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:18.943780+0000","last_became_peered":"2024-09-11T01:14:18.943780+0000","last_unstale":"2024-09-11T01:14:24.463593+0000","last_undegraded":"2024-09-11T01:14:18.938570+0000","last_fullsized":"2024-09-11T01:14:18.938451+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":47,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463419+0000","last_change":"2024-09-11T01:14:26.010597+0000","last_active":"2024-09-11T01:14:26.010597+0000","last_peered":"2024-09-11T01:14:24.463419+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:18.942482+0000","last_became_peered":"2024-09-11T01:14:18.942482+0000","last_unstale":"2024-09-11T01:14:24.463419+0000","last_undegraded":"2024-09-11T01:14:18.937604+0000","last_fullsized":"2024-09-11T01:14:18.937484+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":42,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463362+0000","last_change":"2024-09-11T01:14:26.010719+0000","last_active":"2024-09-11T01:14:26.010719+0000","last_peered":"2024-09-11T01:14:24.463362+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:18.942549+0000","last_became_peered":"2024-09-11T01:14:18.942549+0000","last_unstale":"2024-09-11T01:14:24.463362+0000","last_undegraded":"2024-09-11T01:14:18.937978+0000","last_fullsized":"2024-09-11T01:14:18.937869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":58,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463006+0000","last_change":"2024-09-11T01:14:26.011017+0000","last_active":"2024-09-11T01:14:26.011017+0000","last_peered":"2024-09-11T01:14:24.463006+0000","last_clean":"2024-09-11T01:13:50.681364+0000","last_became_active":"2024-09-11T01:14:18.941204+0000","last_became_peered":"2024-09-11T01:14:18.941204+0000","last_unstale":"2024-09-11T01:14:24.463006+0000","last_undegraded":"2024-09-11T01:14:24.463006+0000","last_fullsized":"2024-09-11T01:14:18.936791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":6,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:26.022412+0000","last_change":"2024-09-11T01:14:26.022412+0000","last_active":"2024-09-11T01:14:26.022412+0000","last_peered":"2024-09-11T01:14:24.463826+0000","last_clean":"2024-09-11T01:14:00.657239+0000","last_became_active":"2024-09-11T01:14:18.942988+0000","last_became_peered":"2024-09-11T01:14:18.942988+0000","last_unstale":"2024-09-11T01:14:26.022412+0000","last_undegraded":"2024-09-11T01:14:26.022412+0000","last_fullsized":"2024-09-11T01:14:26.022412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:26.014712+0000","last_change":"2024-09-11T01:14:24.468221+0000","last_active":"2024-09-11T01:14:24.468221+0000","last_peered":"2024-09-11T01:13:44.542790+0000","last_clean":"2024-09-11T01:13:44.542790+0000","last_became_active":"2024-09-11T01:13:36.502294+0000","last_became_peered":"2024-09-11T01:13:36.502294+0000","last_unstale":"2024-09-11T01:14:26.014712+0000","last_undegraded":"2024-09-11T01:14:26.014712+0000","last_fullsized":"2024-09-11T01:14:26.014712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043864000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":43,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463467+0000","last_change":"2024-09-11T01:14:26.012271+0000","last_active":"2024-09-11T01:14:26.012271+0000","last_peered":"2024-09-11T01:14:24.463467+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:18.943718+0000","last_became_peered":"2024-09-11T01:14:18.943718+0000","last_unstale":"2024-09-11T01:14:24.463467+0000","last_undegraded":"2024-09-11T01:14:18.938677+0000","last_fullsized":"2024-09-11T01:14:18.938552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":50,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463179+0000","last_change":"2024-09-11T01:14:26.009930+0000","last_active":"2024-09-11T01:14:26.009930+0000","last_peered":"2024-09-11T01:14:24.463179+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:18.942906+0000","last_became_peered":"2024-09-11T01:14:18.942906+0000","last_unstale":"2024-09-11T01:14:24.463179+0000","last_undegraded":"2024-09-11T01:14:18.938237+0000","last_fullsized":"2024-09-11T01:14:18.938161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":13,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:24.463148+0000","last_change":"2024-09-11T01:14:26.016483+0000","last_active":"2024-09-11T01:14:26.016483+0000","last_peered":"2024-09-11T01:14:24.463148+0000","last_clean":"2024-09-11T01:13:59.881324+0000","last_became_active":"2024-09-11T01:14:18.942225+0000","last_became_peered":"2024-09-11T01:14:18.942225+0000","last_unstale":"2024-09-11T01:14:24.463148+0000","last_undegraded":"2024-09-11T01:14:18.937245+0000","last_fullsized":"2024-09-11T01:14:18.937121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":9,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:24.463819+0000","last_change":"2024-09-11T01:14:26.469334+0000","last_active":"2024-09-11T01:14:26.469334+0000","last_peered":"2024-09-11T01:14:24.463819+0000","last_clean":"2024-09-11T01:13:47.636183+0000","last_became_active":"2024-09-11T01:14:18.942827+0000","last_became_peered":"2024-09-11T01:14:18.942827+0000","last_unstale":"2024-09-11T01:14:24.463819+0000","last_undegraded":"2024-09-11T01:14:18.938800+0000","last_fullsized":"2024-09-11T01:14:18.938724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":13,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:24.463103+0000","last_change":"2024-09-11T01:14:26.021803+0000","last_active":"2024-09-11T01:14:26.021803+0000","last_peered":"2024-09-11T01:14:24.463103+0000","last_clean":"2024-09-11T01:14:00.292751+0000","last_became_active":"2024-09-11T01:14:18.941782+0000","last_became_peered":"2024-09-11T01:14:18.941782+0000","last_unstale":"2024-09-11T01:14:24.463103+0000","last_undegraded":"2024-09-11T01:14:18.936952+0000","last_fullsized":"2024-09-11T01:14:18.936803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":43,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.464063+0000","last_change":"2024-09-11T01:14:26.012478+0000","last_active":"2024-09-11T01:14:26.012478+0000","last_peered":"2024-09-11T01:14:24.464063+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:18.942649+0000","last_became_peered":"2024-09-11T01:14:18.942649+0000","last_unstale":"2024-09-11T01:14:24.464063+0000","last_undegraded":"2024-09-11T01:14:18.938578+0000","last_fullsized":"2024-09-11T01:14:18.938453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":58,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.464280+0000","last_change":"2024-09-11T01:14:26.010244+0000","last_active":"2024-09-11T01:14:26.010244+0000","last_peered":"2024-09-11T01:14:24.464280+0000","last_clean":"2024-09-11T01:13:50.681737+0000","last_became_active":"2024-09-11T01:14:18.942276+0000","last_became_peered":"2024-09-11T01:14:18.942276+0000","last_unstale":"2024-09-11T01:14:24.464280+0000","last_undegraded":"2024-09-11T01:14:24.464280+0000","last_fullsized":"2024-09-11T01:14:18.937322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":48,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.464241+0000","last_change":"2024-09-11T01:14:26.012295+0000","last_active":"2024-09-11T01:14:26.012295+0000","last_peered":"2024-09-11T01:14:24.464241+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:18.942568+0000","last_became_peered":"2024-09-11T01:14:18.942568+0000","last_unstale":"2024-09-11T01:14:24.464241+0000","last_undegraded":"2024-09-11T01:14:18.937932+0000","last_fullsized":"2024-09-11T01:14:18.937812+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:26.020593+0000","last_change":"2024-09-11T01:14:24.468174+0000","last_active":"2024-09-11T01:14:24.468174+0000","last_peered":"2024-09-11T01:13:44.542854+0000","last_clean":"2024-09-11T01:13:44.542854+0000","last_became_active":"2024-09-11T01:13:36.502309+0000","last_became_peered":"2024-09-11T01:13:36.502309+0000","last_unstale":"2024-09-11T01:14:26.020593+0000","last_undegraded":"2024-09-11T01:14:26.020593+0000","last_fullsized":"2024-09-11T01:14:26.020593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045026199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":44,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463730+0000","last_change":"2024-09-11T01:14:26.011816+0000","last_active":"2024-09-11T01:14:26.011816+0000","last_peered":"2024-09-11T01:14:24.463730+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:18.941718+0000","last_became_peered":"2024-09-11T01:14:18.941718+0000","last_unstale":"2024-09-11T01:14:24.463730+0000","last_undegraded":"2024-09-11T01:14:18.937847+0000","last_fullsized":"2024-09-11T01:14:18.937749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":58,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.463241+0000","last_change":"2024-09-11T01:14:26.010612+0000","last_active":"2024-09-11T01:14:26.010612+0000","last_peered":"2024-09-11T01:14:24.463241+0000","last_clean":"2024-09-11T01:13:50.682120+0000","last_became_active":"2024-09-11T01:14:18.941364+0000","last_became_peered":"2024-09-11T01:14:18.941364+0000","last_unstale":"2024-09-11T01:14:24.463241+0000","last_undegraded":"2024-09-11T01:14:24.463241+0000","last_fullsized":"2024-09-11T01:14:18.937184+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":5,"reported_epoch":31,"state":"peering","last_fresh":"2024-09-11T01:14:24.463950+0000","last_change":"2024-09-11T01:14:26.469381+0000","last_active":"2024-09-11T01:14:26.469381+0000","last_peered":"2024-09-11T01:14:24.463950+0000","last_clean":"2024-09-11T01:13:50.687188+0000","last_became_active":"2024-09-11T01:14:18.942849+0000","last_became_peered":"2024-09-11T01:14:18.942849+0000","last_unstale":"2024-09-11T01:14:24.463950+0000","last_undegraded":"2024-09-11T01:14:24.463950+0000","last_fullsized":"2024-09-11T01:14:18.938205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":53,"reported_epoch":30,"state":"peering","last_fresh":"2024-09-11T01:14:24.464067+0000","last_change":"2024-09-11T01:14:26.010823+0000","last_active":"2024-09-11T01:14:26.010823+0000","last_peered":"2024-09-11T01:14:24.464067+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:18.941284+0000","last_became_peered":"2024-09-11T01:14:18.941284+0000","last_unstale":"2024-09-11T01:14:24.464067+0000","last_undegraded":"2024-09-11T01:14:18.937100+0000","last_fullsized":"2024-09-11T01:14:18.936970+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986178,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":256,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":262144,"data_stored":80273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":32435886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182406,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31928,"kb_used_data":292,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339912,"statfs":{"total":96636764160,"available":96604069888,"internally_reserved":0,"allocated":299008,"data_stored":81674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":32370346},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:29.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:30.260 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:30.260 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":70,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017266,"num_in_osds":2,"osd_in_since":1726017266,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":24}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":65421312,"bytes_avail":193208107008,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-11T01:14:26.405988+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:14:30.273 INFO:tasks.lost_unfound.ceph_manager:{'fsid': 'bcb624ec-d02e-4f87-a117-bb3a3b752391', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 4, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 70, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 32, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017266, 'num_in_osds': 2, 'osd_in_since': 1726017266, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 24}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 65421312, 'bytes_avail': 193208107008, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:13:19:552800+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-09-11T01:14:26.405988+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T01:14:30.491 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:30.489+0000 7f7c50649640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-11T01:14:33.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:33.584 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:33.585 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:33.596 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-09-11T01:14:32.406973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":63924,"kb_used_data":572,"kb_used_omap":8,"kb_used_meta":63287,"kb_avail":188679756,"statfs":{"total":193273528320,"available":193208070144,"internally_reserved":0,"allocated":585728,"data_stored":170823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":64806232},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.945461"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":43,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.029087+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:27.029087+0000","last_peered":"2024-09-11T01:14:27.029087+0000","last_clean":"2024-09-11T01:14:27.029087+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:27.029087+0000","last_undegraded":"2024-09-11T01:14:27.029087+0000","last_fullsized":"2024-09-11T01:14:27.029087+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":21,"reported_epoch":32,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:14:27.033139+0000","last_change":"2024-09-11T01:14:27.033384+0000","last_active":"2024-09-11T01:14:27.033139+0000","last_peered":"2024-09-11T01:14:27.033139+0000","last_clean":"2024-09-11T01:14:00.035698+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:27.033139+0000","last_undegraded":"2024-09-11T01:14:27.022152+0000","last_fullsized":"2024-09-11T01:14:27.033139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":14,"reported_epoch":32,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:14:27.033657+0000","last_change":"2024-09-11T01:14:27.033806+0000","last_active":"2024-09-11T01:14:27.033657+0000","last_peered":"2024-09-11T01:14:27.033657+0000","last_clean":"2024-09-11T01:14:00.605389+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:27.033657+0000","last_undegraded":"2024-09-11T01:14:27.023034+0000","last_fullsized":"2024-09-11T01:14:27.033657+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028135+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:27.028135+0000","last_peered":"2024-09-11T01:14:27.028135+0000","last_clean":"2024-09-11T01:14:27.028135+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:27.028135+0000","last_undegraded":"2024-09-11T01:14:27.028135+0000","last_fullsized":"2024-09-11T01:14:27.028135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":51,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.029294+0000","last_change":"2024-09-11T01:14:27.029465+0000","last_active":"2024-09-11T01:14:27.029294+0000","last_peered":"2024-09-11T01:14:27.029294+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:27.029294+0000","last_undegraded":"2024-09-11T01:14:27.019859+0000","last_fullsized":"2024-09-11T01:14:27.029294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":52,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.030406+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:27.030406+0000","last_peered":"2024-09-11T01:14:27.030406+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:27.030406+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:27.030406+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":53,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.029252+0000","last_change":"2024-09-11T01:14:27.029390+0000","last_active":"2024-09-11T01:14:27.029252+0000","last_peered":"2024-09-11T01:14:27.029252+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:27.029252+0000","last_undegraded":"2024-09-11T01:14:27.019433+0000","last_fullsized":"2024-09-11T01:14:27.029252+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":49,"reported_epoch":32,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:14:27.028506+0000","last_change":"2024-09-11T01:14:27.031816+0000","last_active":"2024-09-11T01:14:27.028506+0000","last_peered":"2024-09-11T01:14:27.028506+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:27.028506+0000","last_undegraded":"2024-09-11T01:14:27.019285+0000","last_fullsized":"2024-09-11T01:14:27.028506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":14,"reported_epoch":32,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:14:27.032663+0000","last_change":"2024-09-11T01:14:27.032916+0000","last_active":"2024-09-11T01:14:27.032663+0000","last_peered":"2024-09-11T01:14:27.032663+0000","last_clean":"2024-09-11T01:14:00.657239+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:27.032663+0000","last_undegraded":"2024-09-11T01:14:27.021698+0000","last_fullsized":"2024-09-11T01:14:27.032663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":49,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.030836+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:27.030836+0000","last_peered":"2024-09-11T01:14:27.030836+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:27.030836+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:27.030836+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":21,"reported_epoch":32,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:14:27.032143+0000","last_change":"2024-09-11T01:14:27.032403+0000","last_active":"2024-09-11T01:14:27.032143+0000","last_peered":"2024-09-11T01:14:27.032143+0000","last_clean":"2024-09-11T01:13:59.881324+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:27.032143+0000","last_undegraded":"2024-09-11T01:14:27.021074+0000","last_fullsized":"2024-09-11T01:14:27.032143+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":21,"reported_epoch":32,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:14:27.031550+0000","last_change":"2024-09-11T01:14:27.031734+0000","last_active":"2024-09-11T01:14:27.031550+0000","last_peered":"2024-09-11T01:14:27.031550+0000","last_clean":"2024-09-11T01:14:00.292751+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:27.031550+0000","last_undegraded":"2024-09-11T01:14:27.021110+0000","last_fullsized":"2024-09-11T01:14:27.031550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":49,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.030299+0000","last_change":"2024-09-11T01:14:27.030470+0000","last_active":"2024-09-11T01:14:27.030299+0000","last_peered":"2024-09-11T01:14:27.030299+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:27.030299+0000","last_undegraded":"2024-09-11T01:14:27.020390+0000","last_fullsized":"2024-09-11T01:14:27.030299+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":54,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.029965+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:27.029965+0000","last_peered":"2024-09-11T01:14:27.029965+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:27.029965+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:27.029965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":50,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.030353+0000","last_change":"2024-09-11T01:14:27.030519+0000","last_active":"2024-09-11T01:14:27.030353+0000","last_peered":"2024-09-11T01:14:27.030353+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:27.030353+0000","last_undegraded":"2024-09-11T01:14:27.019380+0000","last_fullsized":"2024-09-11T01:14:27.030353+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":59,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.030333+0000","last_change":"2024-09-11T01:14:27.030507+0000","last_active":"2024-09-11T01:14:27.030333+0000","last_peered":"2024-09-11T01:14:27.030333+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:27.030333+0000","last_undegraded":"2024-09-11T01:14:27.019759+0000","last_fullsized":"2024-09-11T01:14:27.030333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":264,"kb_used_omap":4,"kb_used_meta":31675,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":270336,"data_stored":82987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":32435886},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182407,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":308,"kb_used_omap":4,"kb_used_meta":31611,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":315392,"data_stored":87836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":32370346},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:33.597 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:33.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:33.968 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 10 pgs degraded","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017266,"num_in_osds":2,"osd_in_since":1726017266,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":5},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":65458176,"bytes_avail":193208070144,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T01:14:30.406665+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:14:33.982 INFO:tasks.lost_unfound.ceph_manager:{'fsid': 'bcb624ec-d02e-4f87-a117-bb3a3b752391', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 10 pgs degraded', 'count': 10}, 'muted': False}}, 'mutes': []}, 'election_epoch': 4, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 74, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 32, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017266, 'num_in_osds': 2, 'osd_in_since': 1726017266, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 5}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 65458176, 'bytes_avail': 193208070144, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:13:19:552800+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T01:14:30.406665+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T01:14:33.983 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-11T01:14:33.983 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:14:33.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:14:34.122 INFO:teuthology.orchestra.run.smithi039.stdout:107374182408 2024-09-11T01:14:34.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:34.127 INFO:teuthology.orchestra.run.smithi039.stdout:133143986180 2024-09-11T01:14:34.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:34.430 INFO:teuthology.orchestra.run.smithi039.stdout:107374182407 2024-09-11T01:14:34.444 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182408 got 107374182407 for osd.0 2024-09-11T01:14:34.477 INFO:teuthology.orchestra.run.smithi039.stdout:133143986179 2024-09-11T01:14:34.489 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986180 got 133143986179 for osd.2 2024-09-11T01:14:35.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:35.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:35.790 INFO:teuthology.orchestra.run.smithi039.stdout:107374182408 2024-09-11T01:14:35.804 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182408 got 107374182408 for osd.0 2024-09-11T01:14:35.804 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:35.823 INFO:teuthology.orchestra.run.smithi039.stdout:133143986180 2024-09-11T01:14:35.839 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986180 got 133143986180 for osd.2 2024-09-11T01:14:35.839 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:35.839 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:36.245 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:36.245 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 54/60 objects degraded (90.000%), 10 pgs degraded","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":76,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":32,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017266,"num_in_osds":2,"osd_in_since":1726017266,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":5},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":65458176,"bytes_avail":193208070144,"bytes_total":193273528320,"degraded_objects":54,"degraded_total":60,"degraded_ratio":0.90000000000000002,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T01:14:30.406665+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:14:36.258 INFO:tasks.lost_unfound.ceph_manager:{'fsid': 'bcb624ec-d02e-4f87-a117-bb3a3b752391', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 54/60 objects degraded (90.000%), 10 pgs degraded', 'count': 10}, 'muted': False}}, 'mutes': []}, 'election_epoch': 4, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 76, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 32, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017266, 'num_in_osds': 2, 'osd_in_since': 1726017266, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 5}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 65458176, 'bytes_avail': 193208070144, 'bytes_total': 193273528320, 'degraded_objects': 54, 'degraded_total': 60, 'degraded_ratio': 0.9, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:13:19:552800+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T01:14:30.406665+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T01:14:36.258 INFO:tasks.lost_unfound:there are 27 unfound objects 2024-09-11T01:14:36.258 DEBUG:teuthology.orchestra.run.smithi039:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfound_pool -t 20 bench 240 write' 2024-09-11T01:14:36.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:hints = 1 2024-09-11T01:14:36.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-11T01:14:36.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Object prefix: benchmark_data_smithi039_41775 2024-09-11T01:14:36.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:14:36.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 0 0 0 0 0 0 - 0 2024-09-11T01:14:37.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 1 20 3110 3090 12.0697 12.0703 0.00680492 0.00644221 2024-09-11T01:14:38.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 2 20 6217 6197 12.1027 12.1367 0.00449873 0.00643965 2024-09-11T01:14:39.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 3 20 9145 9125 11.8807 11.4375 0.00961504 0.00656355 2024-09-11T01:14:40.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 4 19 11848 11829 11.5509 10.5625 0.00978439 0.00675396 2024-09-11T01:14:41.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 5 20 14458 14438 11.2788 10.1914 0.0059524 0.00691625 2024-09-11T01:14:42.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 6 20 16958 16938 11.0265 9.76562 0.00856092 0.00707803 2024-09-11T01:14:43.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 7 20 19358 19338 10.7905 9.375 0.00852225 0.00723206 2024-09-11T01:14:44.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 8 20 21666 21646 10.5685 9.01562 0.00994951 0.00738484 2024-09-11T01:14:45.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 9 20 23891 23871 10.3599 8.69141 0.0082961 0.00753347 2024-09-11T01:14:46.262 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:46.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 10 20 26023 26003 10.1566 8.32812 0.0114011 0.00768484 2024-09-11T01:14:46.811 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:46.811 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:46.833 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-11T01:14:46.410973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":92452071,"num_objects":22598,"num_object_clones":0,"num_object_copies":45196,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":54,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":22598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22625,"num_write_kb":90320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22626,"ondisk_log_size":22626,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":330180,"kb_used_data":178020,"kb_used_omap":8,"kb_used_meta":152119,"kb_avail":188413500,"statfs":{"total":193273528320,"available":192935424000,"internally_reserved":0,"allocated":182292480,"data_stored":181877575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":155770200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,6,2,8,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":92438528,"num_objects":22568,"num_object_clones":0,"num_object_copies":45136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22568,"num_write_kb":90272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003500"},"pg_stats":[{"pgid":"2.f","version":"32'1430","reported_seq":1473,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:44.845890+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:44.845890+0000","last_peered":"2024-09-11T01:14:44.845890+0000","last_clean":"2024-09-11T01:14:44.845890+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:44.845890+0000","last_undegraded":"2024-09-11T01:14:44.845890+0000","last_fullsized":"2024-09-11T01:14:44.845890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1430,"log_dups_size":0,"ondisk_log_size":1430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857280,"num_objects":1430,"num_object_clones":0,"num_object_copies":2860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1430,"num_write_kb":5720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'1424","reported_seq":1441,"reported_epoch":32,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:14:44.461530+0000","last_change":"2024-09-11T01:14:36.335207+0000","last_active":"2024-09-11T01:14:44.461530+0000","last_peered":"2024-09-11T01:14:44.461530+0000","last_clean":"2024-09-11T01:14:00.035698+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:44.461530+0000","last_undegraded":"2024-09-11T01:14:27.033384+0000","last_fullsized":"2024-09-11T01:14:44.461530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1424,"log_dups_size":0,"ondisk_log_size":1424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5817286,"num_objects":1422,"num_object_clones":0,"num_object_copies":2844,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1424,"num_write_kb":5684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1422}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'1341","reported_seq":1352,"reported_epoch":32,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:14:44.461523+0000","last_change":"2024-09-11T01:14:36.321552+0000","last_active":"2024-09-11T01:14:44.461523+0000","last_peered":"2024-09-11T01:14:44.461523+0000","last_clean":"2024-09-11T01:14:00.605389+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:44.461523+0000","last_undegraded":"2024-09-11T01:14:27.033806+0000","last_fullsized":"2024-09-11T01:14:44.461523+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1341,"log_dups_size":0,"ondisk_log_size":1341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5480931,"num_objects":1339,"num_object_clones":0,"num_object_copies":2678,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1341,"num_write_kb":5354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1339}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'1388","reported_seq":1409,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:44.467206+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:44.467206+0000","last_peered":"2024-09-11T01:14:44.467206+0000","last_clean":"2024-09-11T01:14:44.467206+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:44.467206+0000","last_undegraded":"2024-09-11T01:14:44.467206+0000","last_fullsized":"2024-09-11T01:14:44.467206+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1388,"log_dups_size":0,"ondisk_log_size":1388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5685248,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1388,"num_write_kb":5552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'1439","reported_seq":1485,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.836803+0000","last_change":"2024-09-11T01:14:27.029465+0000","last_active":"2024-09-11T01:14:44.836803+0000","last_peered":"2024-09-11T01:14:44.836803+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:44.836803+0000","last_undegraded":"2024-09-11T01:14:27.019859+0000","last_fullsized":"2024-09-11T01:14:44.836803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1439,"log_dups_size":0,"ondisk_log_size":1439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5874630,"num_objects":1436,"num_object_clones":0,"num_object_copies":2872,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1439,"num_write_kb":5740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1436}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'1407","reported_seq":1455,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.833495+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:44.833495+0000","last_peered":"2024-09-11T01:14:44.833495+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:44.833495+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:44.833495+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1407,"log_dups_size":0,"ondisk_log_size":1407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5751750,"num_objects":1406,"num_object_clones":0,"num_object_copies":2812,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1407,"num_write_kb":5619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'1490","reported_seq":1539,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.842968+0000","last_change":"2024-09-11T01:14:27.029390+0000","last_active":"2024-09-11T01:14:44.842968+0000","last_peered":"2024-09-11T01:14:44.842968+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:44.842968+0000","last_undegraded":"2024-09-11T01:14:27.019433+0000","last_fullsized":"2024-09-11T01:14:44.842968+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1490,"log_dups_size":0,"ondisk_log_size":1490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6087622,"num_objects":1488,"num_object_clones":0,"num_object_copies":2976,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1490,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1488}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'1450","reported_seq":1498,"reported_epoch":32,"state":"active+recovering","last_fresh":"2024-09-11T01:14:44.835859+0000","last_change":"2024-09-11T01:14:36.321307+0000","last_active":"2024-09-11T01:14:44.835859+0000","last_peered":"2024-09-11T01:14:44.835859+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:44.835859+0000","last_undegraded":"2024-09-11T01:14:44.835859+0000","last_fullsized":"2024-09-11T01:14:44.835859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1450,"log_dups_size":0,"ondisk_log_size":1450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5935587,"num_objects":1450,"num_object_clones":0,"num_object_copies":2900,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'1393","reported_seq":1402,"reported_epoch":32,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:14:44.461412+0000","last_change":"2024-09-11T01:14:36.321193+0000","last_active":"2024-09-11T01:14:44.461412+0000","last_peered":"2024-09-11T01:14:44.461412+0000","last_clean":"2024-09-11T01:14:00.657239+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:44.461412+0000","last_undegraded":"2024-09-11T01:14:27.032916+0000","last_fullsized":"2024-09-11T01:14:44.461412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5686214,"num_objects":1390,"num_object_clones":0,"num_object_copies":2780,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1393,"num_write_kb":5556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1390}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'1409","reported_seq":1450,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.826739+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:44.826739+0000","last_peered":"2024-09-11T01:14:44.826739+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:44.826739+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:44.826739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5747654,"num_objects":1405,"num_object_clones":0,"num_object_copies":2810,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1409,"num_write_kb":5616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1405}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'1377","reported_seq":1395,"reported_epoch":32,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:14:44.464370+0000","last_change":"2024-09-11T01:14:36.320857+0000","last_active":"2024-09-11T01:14:44.464370+0000","last_peered":"2024-09-11T01:14:44.464370+0000","last_clean":"2024-09-11T01:13:59.881324+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:44.464370+0000","last_undegraded":"2024-09-11T01:14:27.032403+0000","last_fullsized":"2024-09-11T01:14:44.464370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1377,"log_dups_size":0,"ondisk_log_size":1377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5628870,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1377,"num_write_kb":5499,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1376}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'1382","reported_seq":1399,"reported_epoch":32,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:14:44.472681+0000","last_change":"2024-09-11T01:14:36.320301+0000","last_active":"2024-09-11T01:14:44.472681+0000","last_peered":"2024-09-11T01:14:44.472681+0000","last_clean":"2024-09-11T01:14:00.292751+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:44.472681+0000","last_undegraded":"2024-09-11T01:14:27.031734+0000","last_fullsized":"2024-09-11T01:14:44.472681+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1382,"log_dups_size":0,"ondisk_log_size":1382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5645254,"num_objects":1380,"num_object_clones":0,"num_object_copies":2760,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1382,"num_write_kb":5515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1380}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'1393","reported_seq":1439,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.848603+0000","last_change":"2024-09-11T01:14:27.030470+0000","last_active":"2024-09-11T01:14:44.848603+0000","last_peered":"2024-09-11T01:14:44.848603+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:44.848603+0000","last_undegraded":"2024-09-11T01:14:27.020390+0000","last_fullsized":"2024-09-11T01:14:44.848603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1393,"log_dups_size":0,"ondisk_log_size":1393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5702115,"num_objects":1393,"num_object_clones":0,"num_object_copies":2786,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1393,"num_write_kb":5569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'1442","reported_seq":1491,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.846115+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:44.846115+0000","last_peered":"2024-09-11T01:14:44.846115+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:44.846115+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:44.846115+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1442,"log_dups_size":0,"ondisk_log_size":1442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5887401,"num_objects":1440,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1442,"num_write_kb":5752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'1406","reported_seq":1454,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.818680+0000","last_change":"2024-09-11T01:14:27.030519+0000","last_active":"2024-09-11T01:14:44.818680+0000","last_peered":"2024-09-11T01:14:44.818680+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:44.818680+0000","last_undegraded":"2024-09-11T01:14:27.019380+0000","last_fullsized":"2024-09-11T01:14:44.818680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1406,"log_dups_size":0,"ondisk_log_size":1406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5751750,"num_objects":1406,"num_object_clones":0,"num_object_copies":2812,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1406,"num_write_kb":5618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1406}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'1452","reported_seq":1502,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:44.839309+0000","last_change":"2024-09-11T01:14:27.030507+0000","last_active":"2024-09-11T01:14:44.839309+0000","last_peered":"2024-09-11T01:14:44.839309+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:44.839309+0000","last_undegraded":"2024-09-11T01:14:27.019759+0000","last_fullsized":"2024-09-11T01:14:44.839309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":1452,"log_dups_size":0,"ondisk_log_size":1452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5912460,"num_objects":1447,"num_object_clones":0,"num_object_copies":2894,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1452,"num_write_kb":5779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":92452052,"num_objects":22596,"num_object_clones":0,"num_object_copies":45192,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":22596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22623,"num_write_kb":90318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":184111104,"data_stored":183973810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22623,"ondisk_log_size":22623,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":163156,"kb_used_data":87044,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94208684,"statfs":{"total":96636764160,"available":96469692416,"internally_reserved":0,"allocated":89133056,"data_stored":88945707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":77917870},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,1,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182411,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":167024,"kb_used_data":90976,"kb_used_omap":4,"kb_used_meta":76027,"kb_avail":94204816,"statfs":{"total":96636764160,"available":96465731584,"internally_reserved":0,"allocated":93159424,"data_stored":92931868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":77852330},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,3,1,5,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93716480,"data_stored":93680350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90279936,"data_stored":90279936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:46.834 INFO:tasks.lost_unfound:no unfound in 2.f 2024-09-11T01:14:46.835 INFO:tasks.lost_unfound:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-11T01:14:46.835 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-11T01:14:47.052 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:47.053 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_4", 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 1224592654, 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'1", 2024-09-11T01:14:47.054 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_6", 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 3101443614, 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:47.055 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'4", 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'2", 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "NotRecovering", 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:47.056 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:47.057 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:47.057 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:47.057 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:47.074 INFO:tasks.lost_unfound:reverting unfound in 2.e on osd.2 2024-09-11T01:14:47.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost revert 2024-09-11T01:14:47.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 11 20 27807 27787 9.86677 6.96875 0.0102215 0.00791103 2024-09-11T01:14:47.313 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:47.337 INFO:tasks.lost_unfound:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-11T01:14:47.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-11T01:14:47.559 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 1, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 1, 2024-09-11T01:14:47.560 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_7", 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 2358579165, 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:47.561 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "NotRecovering", 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:47.562 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:47.563 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:47.579 INFO:tasks.lost_unfound:reverting unfound in 2.d on osd.2 2024-09-11T01:14:47.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost revert 2024-09-11T01:14:47.814 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T01:14:47.836 INFO:tasks.lost_unfound:no unfound in 2.c 2024-09-11T01:14:47.837 INFO:tasks.lost_unfound:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-11T01:14:47.837 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-11T01:14:48.065 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:48.065 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_3", 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:48.066 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 943960171, 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'5", 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_1", 2024-09-11T01:14:48.067 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 2885884443, 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'4", 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'1", 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:48.068 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:48.069 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:48.086 INFO:tasks.lost_unfound:reverting unfound in 2.b on osd.0 2024-09-11T01:14:48.086 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost revert 2024-09-11T01:14:48.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 12 20 29548 29528 9.61123 6.80078 0.00971945 0.00812286 2024-09-11T01:14:48.325 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:48.349 INFO:tasks.lost_unfound:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-11T01:14:48.349 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-11T01:14:48.580 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:48.580 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_9", 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:48.581 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 1217469130, 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'1", 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_9", 2024-09-11T01:14:48.582 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 170713514, 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'2", 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:48.583 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:48.584 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:48.606 INFO:tasks.lost_unfound:reverting unfound in 2.a on osd.0 2024-09-11T01:14:48.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost revert 2024-09-11T01:14:48.841 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:48.865 INFO:tasks.lost_unfound:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-11T01:14:48.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:49.079 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_5", 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 2998549577, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:49.080 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'4", 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'2", 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_2", 2024-09-11T01:14:49.081 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 4032093161, 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:49.082 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'1", 2024-09-11T01:14:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:49.083 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:49.084 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:49.085 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:49.085 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:49.097 INFO:tasks.lost_unfound:reverting unfound in 2.9 on osd.0 2024-09-11T01:14:49.097 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost revert 2024-09-11T01:14:49.312 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 13 20 31112 31092 9.34182 6.10938 0.0145621 0.0083548 2024-09-11T01:14:49.324 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:49.344 INFO:tasks.lost_unfound:listing missing/lost in 2.8 state active+recovering 2024-09-11T01:14:49.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-11T01:14:49.565 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:49.566 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 1, 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 1, 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_8", 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 2730327960, 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:49.567 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'1", 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:49.568 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:49.569 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:49.586 INFO:tasks.lost_unfound:reverting unfound in 2.8 on osd.0 2024-09-11T01:14:49.586 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost revert 2024-09-11T01:14:49.836 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T01:14:49.857 INFO:tasks.lost_unfound:no unfound in 1.0 2024-09-11T01:14:49.857 INFO:tasks.lost_unfound:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-11T01:14:49.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-11T01:14:50.087 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:50.088 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_8", 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 1901787523, 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.089 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'5", 2024-09-11T01:14:50.090 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'3", 2024-09-11T01:14:50.090 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:50.090 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:50.090 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:50.090 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.090 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:50.091 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:50.091 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_1", 2024-09-11T01:14:50.091 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:50.091 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 3122379363, 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'4", 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:50.092 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "NotRecovering", 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:50.093 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:50.108 INFO:tasks.lost_unfound:reverting unfound in 2.3 on osd.2 2024-09-11T01:14:50.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost revert 2024-09-11T01:14:50.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 14 20 32506 32486 9.06346 5.44531 0.0114195 0.00861291 2024-09-11T01:14:50.341 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:50.358 INFO:tasks.lost_unfound:no unfound in 1.1 2024-09-11T01:14:50.358 INFO:tasks.lost_unfound:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-11T01:14:50.359 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-11T01:14:50.587 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:50.587 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:50.587 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:50.587 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:50.587 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:50.588 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:50.588 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:50.588 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:50.588 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:50.588 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.589 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:50.589 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:50.589 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:50.589 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:50.589 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:50.589 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_6", 2024-09-11T01:14:50.590 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:50.590 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 1708673474, 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'5", 2024-09-11T01:14:50.591 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_9", 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 831402034, 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:50.592 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'6", 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:50.593 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:50.594 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:50.594 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:50.594 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:50.606 INFO:tasks.lost_unfound:reverting unfound in 2.2 on osd.0 2024-09-11T01:14:50.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost revert 2024-09-11T01:14:50.696 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:14:50.693+0000 7f7c50649640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-09-11T01:14:50.832 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:50.855 INFO:tasks.lost_unfound:no unfound in 1.2 2024-09-11T01:14:50.855 INFO:tasks.lost_unfound:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-11T01:14:50.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:51.067 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_3", 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 459768081, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:51.068 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'1", 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_3", 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 480221105, 2024-09-11T01:14:51.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'2", 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "NotRecovering", 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:51.070 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:51.071 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:51.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:51.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:51.071 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:51.071 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:51.093 INFO:tasks.lost_unfound:reverting unfound in 2.1 on osd.2 2024-09-11T01:14:51.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost revert 2024-09-11T01:14:51.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 15 20 34106 34086 8.87586 6.25 0.0252786 0.0087929 2024-09-11T01:14:51.331 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:51.354 INFO:tasks.lost_unfound:no unfound in 1.3 2024-09-11T01:14:51.354 INFO:tasks.lost_unfound:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-11T01:14:51.354 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-11T01:14:51.577 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:51.577 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:51.577 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:51.577 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:51.578 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_5", 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 4248945696, 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:51.579 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_5", 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 4212789968, 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'4", 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:51.580 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "NotRecovering", 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:51.581 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:51.602 INFO:tasks.lost_unfound:reverting unfound in 2.0 on osd.2 2024-09-11T01:14:51.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost revert 2024-09-11T01:14:51.849 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:51.867 INFO:tasks.lost_unfound:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-11T01:14:51.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-11T01:14:52.078 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:52.078 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:52.078 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:52.078 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 1, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 1, 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_7", 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:52.079 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 3761017527, 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'1", 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:52.080 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:52.081 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:52.099 INFO:tasks.lost_unfound:reverting unfound in 2.7 on osd.0 2024-09-11T01:14:52.099 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost revert 2024-09-11T01:14:52.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 16 20 35590 35570 8.6834 5.79688 0.0152403 0.00898955 2024-09-11T01:14:52.330 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T01:14:52.353 INFO:tasks.lost_unfound:no unfound in 1.4 2024-09-11T01:14:52.354 INFO:tasks.lost_unfound:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-11T01:14:52.354 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-11T01:14:52.572 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 3, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 3, 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:52.573 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_8", 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 1861424966, 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'5", 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:52.574 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_6", 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 3910131350, 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'4", 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:52.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_4", 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 2787839990, 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:52.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'3", 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:52.577 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:52.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:52.578 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:52.578 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:52.578 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:52.594 INFO:tasks.lost_unfound:reverting unfound in 2.6 on osd.0 2024-09-11T01:14:52.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost revert 2024-09-11T01:14:52.835 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-11T01:14:52.854 INFO:tasks.lost_unfound:no unfound in 1.5 2024-09-11T01:14:52.854 INFO:tasks.lost_unfound:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-11T01:14:52.854 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-11T01:14:53.068 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:53.068 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:53.068 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:53.068 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.068 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 2, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 2, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_1", 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:53.069 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 26281365, 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'1", 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "new_2", 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.070 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 3196340981, 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'2", 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:53.071 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:53.072 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:53.089 INFO:tasks.lost_unfound:reverting unfound in 2.5 on osd.0 2024-09-11T01:14:53.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost revert 2024-09-11T01:14:53.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 17 20 37003 36983 8.49726 5.51953 0.00926077 0.00918716 2024-09-11T01:14:53.324 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:14:53.345 INFO:tasks.lost_unfound:no unfound in 1.6 2024-09-11T01:14:53.345 INFO:tasks.lost_unfound:no unfound in 1.7 2024-09-11T01:14:53.345 INFO:tasks.lost_unfound:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-11T01:14:53.346 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "offset": { 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "", 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": 0, 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 0, 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": -9223372036854775808, 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "num_missing": 4, 2024-09-11T01:14:53.573 INFO:teuthology.orchestra.run.smithi039.stdout: "num_unfound": 3, 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "objects": [ 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_4", 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 873891716, 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'8", 2024-09-11T01:14:53.574 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existing_7", 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 3996155460, 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.575 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'9", 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "21'6", 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": { 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "oid": "existed_2", 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "key": "", 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "snapid": -2, 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "hash": 1767667508, 2024-09-11T01:14:53.576 INFO:teuthology.orchestra.run.smithi039.stdout: "max": 0, 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "pool": 2, 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "namespace": "" 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "need": "23'7", 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "have": "0'0", 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "flags": "none", 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "locations": [] 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "state": "Active", 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "available_might_have_unfound": true, 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: "might_have_unfound": [ 2024-09-11T01:14:53.577 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:14:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "osd": "1", 2024-09-11T01:14:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "status": "osd is down" 2024-09-11T01:14:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:14:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:14:53.578 INFO:teuthology.orchestra.run.smithi039.stdout: "more": false 2024-09-11T01:14:53.578 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:14:53.594 INFO:tasks.lost_unfound:reverting unfound in 2.4 on osd.0 2024-09-11T01:14:53.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost revert 2024-09-11T01:14:53.838 INFO:teuthology.orchestra.run.smithi039.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-11T01:14:53.861 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-11T01:14:54.081 INFO:teuthology.orchestra.run.smithi039.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-11T01:14:54.102 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-11T01:14:54.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 18 20 38402 38382 8.32879 5.46484 0.0205331 0.00937318 2024-09-11T01:14:54.315 INFO:teuthology.orchestra.run.smithi039.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-11T01:14:54.339 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:14:54.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:14:54.593 INFO:teuthology.orchestra.run.smithi039.stdout:133143986186 2024-09-11T01:14:54.594 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:54.599 INFO:teuthology.orchestra.run.smithi039.stdout:107374182413 2024-09-11T01:14:54.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:55.168 INFO:teuthology.orchestra.run.smithi039.stdout:133143986184 2024-09-11T01:14:55.188 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986184 for osd.2 2024-09-11T01:14:55.202 INFO:teuthology.orchestra.run.smithi039.stdout:107374182412 2024-09-11T01:14:55.225 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182413 got 107374182412 for osd.0 2024-09-11T01:14:55.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 19 20 39741 39721 8.1657 5.23047 0.0155535 0.00956232 2024-09-11T01:14:56.189 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:56.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:14:56.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:14:56.312615+0000 min lat: 0.00228443 max lat: 0.125246 avg lat: 0.00966097 2024-09-11T01:14:56.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:14:56.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 20 20 41398 41378 8.08102 6.47266 0.0190029 0.00966097 2024-09-11T01:14:56.731 INFO:teuthology.orchestra.run.smithi039.stdout:133143986184 2024-09-11T01:14:56.760 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986184 for osd.2 2024-09-11T01:14:56.792 INFO:teuthology.orchestra.run.smithi039.stdout:107374182414 2024-09-11T01:14:56.811 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182413 got 107374182414 for osd.0 2024-09-11T01:14:56.812 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:57.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 21 20 42806 42786 7.95809 5.5 0.00969358 0.00981119 2024-09-11T01:14:57.760 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:14:58.308 INFO:teuthology.orchestra.run.smithi039.stdout:133143986186 2024-09-11T01:14:58.313 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 22 20 44325 44305 7.8659 5.93359 0.0126232 0.00992532 2024-09-11T01:14:58.330 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986186 for osd.2 2024-09-11T01:14:58.331 DEBUG:teuthology.parallel:result is None 2024-09-11T01:14:58.331 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-11T01:14:58.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:58.888 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:58.889 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:58.909 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-11T01:14:58.414398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":159665016,"num_objects":39003,"num_object_clones":0,"num_object_copies":78006,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":39003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28476,"num_write_kb":113784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39063,"ondisk_log_size":39063,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":513464,"kb_used_data":309912,"kb_used_omap":8,"kb_used_meta":203511,"kb_avail":188230216,"statfs":{"total":193273528320,"available":192747741184,"internally_reserved":0,"allocated":317349888,"data_stored":316920531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":208395608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,6,6,16,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67212945,"num_objects":16405,"num_object_clones":0,"num_object_copies":32810,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":16405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":23464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.f","version":"32'2430","reported_seq":2473,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.848979+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:54.848979+0000","last_peered":"2024-09-11T01:14:54.848979+0000","last_clean":"2024-09-11T01:14:54.848979+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:54.848979+0000","last_undegraded":"2024-09-11T01:14:54.848979+0000","last_fullsized":"2024-09-11T01:14:54.848979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2430,"log_dups_size":0,"ondisk_log_size":2430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9953280,"num_objects":2430,"num_object_clones":0,"num_object_copies":4860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2430,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'2447","reported_seq":2501,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.537553+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:14:54.537553+0000","last_peered":"2024-09-11T01:14:54.537553+0000","last_clean":"2024-09-11T01:14:54.537553+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:54.537553+0000","last_undegraded":"2024-09-11T01:14:54.537553+0000","last_fullsized":"2024-09-11T01:14:54.537553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":2447,"log_dups_size":0,"ondisk_log_size":2447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9999302,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":545,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'2377","reported_seq":2426,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.556750+0000","last_change":"2024-09-11T01:14:50.131076+0000","last_active":"2024-09-11T01:14:54.556750+0000","last_peered":"2024-09-11T01:14:54.556750+0000","last_clean":"2024-09-11T01:14:54.556750+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:54.556750+0000","last_undegraded":"2024-09-11T01:14:54.556750+0000","last_fullsized":"2024-09-11T01:14:54.556750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:54.197115+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":438,"num_write_kb":1752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'2439","reported_seq":2460,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.544471+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:54.544471+0000","last_peered":"2024-09-11T01:14:54.544471+0000","last_clean":"2024-09-11T01:14:54.544471+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:54.544471+0000","last_undegraded":"2024-09-11T01:14:54.544471+0000","last_fullsized":"2024-09-11T01:14:54.544471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2439,"log_dups_size":0,"ondisk_log_size":2439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9990144,"num_objects":2439,"num_object_clones":0,"num_object_copies":4878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'2455","reported_seq":2500,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.836980+0000","last_change":"2024-09-11T01:14:27.029465+0000","last_active":"2024-09-11T01:14:54.836980+0000","last_peered":"2024-09-11T01:14:54.836980+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:54.836980+0000","last_undegraded":"2024-09-11T01:14:27.019859+0000","last_fullsized":"2024-09-11T01:14:54.836980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2455,"log_dups_size":0,"ondisk_log_size":2455,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10027974,"num_objects":2450,"num_object_clones":0,"num_object_copies":4900,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2453,"num_write_kb":9796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'2394","reported_seq":2442,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.840466+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:54.840466+0000","last_peered":"2024-09-11T01:14:54.840466+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:54.840466+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:54.840466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2394,"log_dups_size":0,"ondisk_log_size":2394,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9786310,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2392,"num_write_kb":9559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'2514","reported_seq":2562,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.827717+0000","last_change":"2024-09-11T01:14:27.029390+0000","last_active":"2024-09-11T01:14:54.827717+0000","last_peered":"2024-09-11T01:14:54.827717+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:54.827717+0000","last_undegraded":"2024-09-11T01:14:27.019433+0000","last_fullsized":"2024-09-11T01:14:54.827717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10273734,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2508}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'2446","reported_seq":2493,"reported_epoch":32,"state":"active+recovering","last_fresh":"2024-09-11T01:14:54.841656+0000","last_change":"2024-09-11T01:14:36.321307+0000","last_active":"2024-09-11T01:14:54.841656+0000","last_peered":"2024-09-11T01:14:54.841656+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:54.841656+0000","last_undegraded":"2024-09-11T01:14:54.841656+0000","last_fullsized":"2024-09-11T01:14:54.841656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2446,"log_dups_size":0,"ondisk_log_size":2446,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10011107,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2445,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'2413","reported_seq":2462,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.550982+0000","last_change":"2024-09-11T01:14:51.929341+0000","last_active":"2024-09-11T01:14:54.550982+0000","last_peered":"2024-09-11T01:14:54.550982+0000","last_clean":"2024-09-11T01:14:54.550982+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:54.550982+0000","last_undegraded":"2024-09-11T01:14:54.550982+0000","last_fullsized":"2024-09-11T01:14:54.550982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":2413,"log_dups_size":0,"ondisk_log_size":2413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:29.457390+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9855459,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'2433","reported_seq":2474,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.851615+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:54.851615+0000","last_peered":"2024-09-11T01:14:54.851615+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:54.851615+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:54.851615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9933766,"num_objects":2427,"num_object_clones":0,"num_object_copies":4854,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2427}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'2391","reported_seq":2452,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.510360+0000","last_change":"2024-09-11T01:14:53.159504+0000","last_active":"2024-09-11T01:14:54.510360+0000","last_peered":"2024-09-11T01:14:54.510360+0000","last_clean":"2024-09-11T01:14:54.510360+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:54.510360+0000","last_undegraded":"2024-09-11T01:14:54.510360+0000","last_fullsized":"2024-09-11T01:14:54.510360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:11.511441+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9773539,"num_objects":2387,"num_object_clones":0,"num_object_copies":4774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'2449","reported_seq":2510,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.521227+0000","last_change":"2024-09-11T01:14:54.063365+0000","last_active":"2024-09-11T01:14:54.521227+0000","last_peered":"2024-09-11T01:14:54.521227+0000","last_clean":"2024-09-11T01:14:54.521227+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:54.521227+0000","last_undegraded":"2024-09-11T01:14:54.521227+0000","last_fullsized":"2024-09-11T01:14:54.521227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:19.905343+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10006528,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'2444","reported_seq":2491,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.849098+0000","last_change":"2024-09-11T01:14:27.030470+0000","last_active":"2024-09-11T01:14:54.849098+0000","last_peered":"2024-09-11T01:14:54.849098+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:54.849098+0000","last_undegraded":"2024-09-11T01:14:27.020390+0000","last_fullsized":"2024-09-11T01:14:54.849098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002915,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'2459","reported_seq":2506,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.826963+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:54.826963+0000","last_peered":"2024-09-11T01:14:54.826963+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:54.826963+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:54.826963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2459,"log_dups_size":0,"ondisk_log_size":2459,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10040745,"num_objects":2454,"num_object_clones":0,"num_object_copies":4908,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2456,"num_write_kb":9808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'2503","reported_seq":2550,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.840468+0000","last_change":"2024-09-11T01:14:27.030519+0000","last_active":"2024-09-11T01:14:54.840468+0000","last_peered":"2024-09-11T01:14:54.840468+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:54.840468+0000","last_undegraded":"2024-09-11T01:14:27.019380+0000","last_fullsized":"2024-09-11T01:14:54.840468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10236870,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2501,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2501}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'2466","reported_seq":2512,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.858884+0000","last_change":"2024-09-11T01:14:27.030507+0000","last_active":"2024-09-11T01:14:54.858884+0000","last_peered":"2024-09-11T01:14:54.858884+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:54.858884+0000","last_undegraded":"2024-09-11T01:14:27.019759+0000","last_fullsized":"2024-09-11T01:14:54.858884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2466,"log_dups_size":0,"ondisk_log_size":2466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10053516,"num_objects":2458,"num_object_clones":0,"num_object_copies":4916,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":159664997,"num_objects":39001,"num_object_clones":0,"num_object_copies":78002,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":39001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28474,"num_write_kb":113782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319094784,"data_stored":318943038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39060,"ondisk_log_size":39060,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253160,"kb_used_data":153432,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94118680,"statfs":{"total":96636764160,"available":96377528320,"internally_reserved":0,"allocated":157114368,"data_stored":156912567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260304,"kb_used_data":156480,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94111536,"statfs":{"total":96636764160,"available":96370212864,"internally_reserved":0,"allocated":160235520,"data_stored":160007964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,3,13],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":160317440,"data_stored":160281310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158662656,"data_stored":158648204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:58.911 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:59.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 23 20 45720 45700 7.76081 5.44922 0.00973323 0.0100591 2024-09-11T01:14:59.458 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:59.459 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:59.481 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-11T01:14:58.414398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":159665016,"num_objects":39003,"num_object_clones":0,"num_object_copies":78006,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":39003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28476,"num_write_kb":113784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39063,"ondisk_log_size":39063,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":513464,"kb_used_data":309912,"kb_used_omap":8,"kb_used_meta":203511,"kb_avail":188230216,"statfs":{"total":193273528320,"available":192747741184,"internally_reserved":0,"allocated":317349888,"data_stored":316920531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":208395608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,6,6,16,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67212945,"num_objects":16405,"num_object_clones":0,"num_object_copies":32810,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":16405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":23464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.f","version":"32'2430","reported_seq":2473,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.848979+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:54.848979+0000","last_peered":"2024-09-11T01:14:54.848979+0000","last_clean":"2024-09-11T01:14:54.848979+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:54.848979+0000","last_undegraded":"2024-09-11T01:14:54.848979+0000","last_fullsized":"2024-09-11T01:14:54.848979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2430,"log_dups_size":0,"ondisk_log_size":2430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9953280,"num_objects":2430,"num_object_clones":0,"num_object_copies":4860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2430,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'2447","reported_seq":2501,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.537553+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:14:54.537553+0000","last_peered":"2024-09-11T01:14:54.537553+0000","last_clean":"2024-09-11T01:14:54.537553+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:54.537553+0000","last_undegraded":"2024-09-11T01:14:54.537553+0000","last_fullsized":"2024-09-11T01:14:54.537553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":2447,"log_dups_size":0,"ondisk_log_size":2447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9999302,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":545,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'2377","reported_seq":2426,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.556750+0000","last_change":"2024-09-11T01:14:50.131076+0000","last_active":"2024-09-11T01:14:54.556750+0000","last_peered":"2024-09-11T01:14:54.556750+0000","last_clean":"2024-09-11T01:14:54.556750+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:54.556750+0000","last_undegraded":"2024-09-11T01:14:54.556750+0000","last_fullsized":"2024-09-11T01:14:54.556750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:54.197115+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":438,"num_write_kb":1752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'2439","reported_seq":2460,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.544471+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:54.544471+0000","last_peered":"2024-09-11T01:14:54.544471+0000","last_clean":"2024-09-11T01:14:54.544471+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:54.544471+0000","last_undegraded":"2024-09-11T01:14:54.544471+0000","last_fullsized":"2024-09-11T01:14:54.544471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2439,"log_dups_size":0,"ondisk_log_size":2439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9990144,"num_objects":2439,"num_object_clones":0,"num_object_copies":4878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'2455","reported_seq":2500,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.836980+0000","last_change":"2024-09-11T01:14:27.029465+0000","last_active":"2024-09-11T01:14:54.836980+0000","last_peered":"2024-09-11T01:14:54.836980+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:54.836980+0000","last_undegraded":"2024-09-11T01:14:27.019859+0000","last_fullsized":"2024-09-11T01:14:54.836980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2455,"log_dups_size":0,"ondisk_log_size":2455,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10027974,"num_objects":2450,"num_object_clones":0,"num_object_copies":4900,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2453,"num_write_kb":9796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'2394","reported_seq":2442,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.840466+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:54.840466+0000","last_peered":"2024-09-11T01:14:54.840466+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:54.840466+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:54.840466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2394,"log_dups_size":0,"ondisk_log_size":2394,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9786310,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2392,"num_write_kb":9559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'2514","reported_seq":2562,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.827717+0000","last_change":"2024-09-11T01:14:27.029390+0000","last_active":"2024-09-11T01:14:54.827717+0000","last_peered":"2024-09-11T01:14:54.827717+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:54.827717+0000","last_undegraded":"2024-09-11T01:14:27.019433+0000","last_fullsized":"2024-09-11T01:14:54.827717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10273734,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2508}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'2446","reported_seq":2493,"reported_epoch":32,"state":"active+recovering","last_fresh":"2024-09-11T01:14:54.841656+0000","last_change":"2024-09-11T01:14:36.321307+0000","last_active":"2024-09-11T01:14:54.841656+0000","last_peered":"2024-09-11T01:14:54.841656+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:54.841656+0000","last_undegraded":"2024-09-11T01:14:54.841656+0000","last_fullsized":"2024-09-11T01:14:54.841656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2446,"log_dups_size":0,"ondisk_log_size":2446,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10011107,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2445,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'2413","reported_seq":2462,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.550982+0000","last_change":"2024-09-11T01:14:51.929341+0000","last_active":"2024-09-11T01:14:54.550982+0000","last_peered":"2024-09-11T01:14:54.550982+0000","last_clean":"2024-09-11T01:14:54.550982+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:54.550982+0000","last_undegraded":"2024-09-11T01:14:54.550982+0000","last_fullsized":"2024-09-11T01:14:54.550982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":2413,"log_dups_size":0,"ondisk_log_size":2413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:29.457390+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9855459,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'2433","reported_seq":2474,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.851615+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:54.851615+0000","last_peered":"2024-09-11T01:14:54.851615+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:54.851615+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:54.851615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9933766,"num_objects":2427,"num_object_clones":0,"num_object_copies":4854,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2427}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'2391","reported_seq":2452,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.510360+0000","last_change":"2024-09-11T01:14:53.159504+0000","last_active":"2024-09-11T01:14:54.510360+0000","last_peered":"2024-09-11T01:14:54.510360+0000","last_clean":"2024-09-11T01:14:54.510360+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:54.510360+0000","last_undegraded":"2024-09-11T01:14:54.510360+0000","last_fullsized":"2024-09-11T01:14:54.510360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:11.511441+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9773539,"num_objects":2387,"num_object_clones":0,"num_object_copies":4774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'2449","reported_seq":2510,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.521227+0000","last_change":"2024-09-11T01:14:54.063365+0000","last_active":"2024-09-11T01:14:54.521227+0000","last_peered":"2024-09-11T01:14:54.521227+0000","last_clean":"2024-09-11T01:14:54.521227+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:54.521227+0000","last_undegraded":"2024-09-11T01:14:54.521227+0000","last_fullsized":"2024-09-11T01:14:54.521227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:19.905343+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10006528,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'2444","reported_seq":2491,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.849098+0000","last_change":"2024-09-11T01:14:27.030470+0000","last_active":"2024-09-11T01:14:54.849098+0000","last_peered":"2024-09-11T01:14:54.849098+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:54.849098+0000","last_undegraded":"2024-09-11T01:14:27.020390+0000","last_fullsized":"2024-09-11T01:14:54.849098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002915,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'2459","reported_seq":2506,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.826963+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:54.826963+0000","last_peered":"2024-09-11T01:14:54.826963+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:54.826963+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:54.826963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2459,"log_dups_size":0,"ondisk_log_size":2459,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10040745,"num_objects":2454,"num_object_clones":0,"num_object_copies":4908,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2456,"num_write_kb":9808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'2503","reported_seq":2550,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.840468+0000","last_change":"2024-09-11T01:14:27.030519+0000","last_active":"2024-09-11T01:14:54.840468+0000","last_peered":"2024-09-11T01:14:54.840468+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:54.840468+0000","last_undegraded":"2024-09-11T01:14:27.019380+0000","last_fullsized":"2024-09-11T01:14:54.840468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10236870,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2501,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2501}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'2466","reported_seq":2512,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.858884+0000","last_change":"2024-09-11T01:14:27.030507+0000","last_active":"2024-09-11T01:14:54.858884+0000","last_peered":"2024-09-11T01:14:54.858884+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:54.858884+0000","last_undegraded":"2024-09-11T01:14:27.019759+0000","last_fullsized":"2024-09-11T01:14:54.858884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2466,"log_dups_size":0,"ondisk_log_size":2466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10053516,"num_objects":2458,"num_object_clones":0,"num_object_copies":4916,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":159664997,"num_objects":39001,"num_object_clones":0,"num_object_copies":78002,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":39001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28474,"num_write_kb":113782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319094784,"data_stored":318943038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39060,"ondisk_log_size":39060,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253160,"kb_used_data":153432,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94118680,"statfs":{"total":96636764160,"available":96377528320,"internally_reserved":0,"allocated":157114368,"data_stored":156912567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260304,"kb_used_data":156480,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94111536,"statfs":{"total":96636764160,"available":96370212864,"internally_reserved":0,"allocated":160235520,"data_stored":160007964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,3,13],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":160317440,"data_stored":160281310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158662656,"data_stored":158648204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:59.483 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:59.867 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:14:59.867 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:14:59.889 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-11T01:14:58.414398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":159665016,"num_objects":39003,"num_object_clones":0,"num_object_copies":78006,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":36,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":39003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28476,"num_write_kb":113784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39063,"ondisk_log_size":39063,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":513464,"kb_used_data":309912,"kb_used_omap":8,"kb_used_meta":203511,"kb_avail":188230216,"statfs":{"total":193273528320,"available":192747741184,"internally_reserved":0,"allocated":317349888,"data_stored":316920531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":208395608},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,6,6,16,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67212945,"num_objects":16405,"num_object_clones":0,"num_object_copies":32810,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":16405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5851,"num_write_kb":23464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003424"},"pg_stats":[{"pgid":"2.f","version":"32'2430","reported_seq":2473,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.848979+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:54.848979+0000","last_peered":"2024-09-11T01:14:54.848979+0000","last_clean":"2024-09-11T01:14:54.848979+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:54.848979+0000","last_undegraded":"2024-09-11T01:14:54.848979+0000","last_fullsized":"2024-09-11T01:14:54.848979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2430,"log_dups_size":0,"ondisk_log_size":2430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9953280,"num_objects":2430,"num_object_clones":0,"num_object_copies":4860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2430,"num_write_kb":9720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'2447","reported_seq":2501,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.537553+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:14:54.537553+0000","last_peered":"2024-09-11T01:14:54.537553+0000","last_clean":"2024-09-11T01:14:54.537553+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:54.537553+0000","last_undegraded":"2024-09-11T01:14:54.537553+0000","last_fullsized":"2024-09-11T01:14:54.537553+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":2447,"log_dups_size":0,"ondisk_log_size":2447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9999302,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":545,"num_write_kb":2180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'2377","reported_seq":2426,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.556750+0000","last_change":"2024-09-11T01:14:50.131076+0000","last_active":"2024-09-11T01:14:54.556750+0000","last_peered":"2024-09-11T01:14:54.556750+0000","last_clean":"2024-09-11T01:14:54.556750+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:54.556750+0000","last_undegraded":"2024-09-11T01:14:54.556750+0000","last_fullsized":"2024-09-11T01:14:54.556750+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:54.197115+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":438,"num_write_kb":1752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'2439","reported_seq":2460,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.544471+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:54.544471+0000","last_peered":"2024-09-11T01:14:54.544471+0000","last_clean":"2024-09-11T01:14:54.544471+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:54.544471+0000","last_undegraded":"2024-09-11T01:14:54.544471+0000","last_fullsized":"2024-09-11T01:14:54.544471+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2439,"log_dups_size":0,"ondisk_log_size":2439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9990144,"num_objects":2439,"num_object_clones":0,"num_object_copies":4878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'2455","reported_seq":2500,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.836980+0000","last_change":"2024-09-11T01:14:27.029465+0000","last_active":"2024-09-11T01:14:54.836980+0000","last_peered":"2024-09-11T01:14:54.836980+0000","last_clean":"2024-09-11T01:13:59.933294+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:54.836980+0000","last_undegraded":"2024-09-11T01:14:27.019859+0000","last_fullsized":"2024-09-11T01:14:54.836980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2455,"log_dups_size":0,"ondisk_log_size":2455,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10027974,"num_objects":2450,"num_object_clones":0,"num_object_copies":4900,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2453,"num_write_kb":9796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2449}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'2394","reported_seq":2442,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.840466+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:54.840466+0000","last_peered":"2024-09-11T01:14:54.840466+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:54.840466+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:54.840466+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2394,"log_dups_size":0,"ondisk_log_size":2394,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9786310,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2392,"num_write_kb":9559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2390}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'2514","reported_seq":2562,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.827717+0000","last_change":"2024-09-11T01:14:27.029390+0000","last_active":"2024-09-11T01:14:54.827717+0000","last_peered":"2024-09-11T01:14:54.827717+0000","last_clean":"2024-09-11T01:14:00.189997+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:54.827717+0000","last_undegraded":"2024-09-11T01:14:27.019433+0000","last_fullsized":"2024-09-11T01:14:54.827717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10273734,"num_objects":2510,"num_object_clones":0,"num_object_copies":5020,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2512,"num_write_kb":10036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2508}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'2446","reported_seq":2493,"reported_epoch":32,"state":"active+recovering","last_fresh":"2024-09-11T01:14:54.841656+0000","last_change":"2024-09-11T01:14:36.321307+0000","last_active":"2024-09-11T01:14:54.841656+0000","last_peered":"2024-09-11T01:14:54.841656+0000","last_clean":"2024-09-11T01:13:48.655841+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:54.841656+0000","last_undegraded":"2024-09-11T01:14:54.841656+0000","last_fullsized":"2024-09-11T01:14:54.841656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2446,"log_dups_size":0,"ondisk_log_size":2446,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10011107,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2445,"num_write_kb":9777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'2413","reported_seq":2462,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.550982+0000","last_change":"2024-09-11T01:14:51.929341+0000","last_active":"2024-09-11T01:14:54.550982+0000","last_peered":"2024-09-11T01:14:54.550982+0000","last_clean":"2024-09-11T01:14:54.550982+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:54.550982+0000","last_undegraded":"2024-09-11T01:14:54.550982+0000","last_fullsized":"2024-09-11T01:14:54.550982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":2413,"log_dups_size":0,"ondisk_log_size":2413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:29.457390+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9855459,"num_objects":2407,"num_object_clones":0,"num_object_copies":4814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":280,"num_write_kb":1120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'2433","reported_seq":2474,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.851615+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:54.851615+0000","last_peered":"2024-09-11T01:14:54.851615+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:54.851615+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:54.851615+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9933766,"num_objects":2427,"num_object_clones":0,"num_object_copies":4854,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2427}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'2391","reported_seq":2452,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.510360+0000","last_change":"2024-09-11T01:14:53.159504+0000","last_active":"2024-09-11T01:14:54.510360+0000","last_peered":"2024-09-11T01:14:54.510360+0000","last_clean":"2024-09-11T01:14:54.510360+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:54.510360+0000","last_undegraded":"2024-09-11T01:14:54.510360+0000","last_fullsized":"2024-09-11T01:14:54.510360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:11.511441+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9773539,"num_objects":2387,"num_object_clones":0,"num_object_copies":4774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":160,"num_write_kb":640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'2449","reported_seq":2510,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.521227+0000","last_change":"2024-09-11T01:14:54.063365+0000","last_active":"2024-09-11T01:14:54.521227+0000","last_peered":"2024-09-11T01:14:54.521227+0000","last_clean":"2024-09-11T01:14:54.521227+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:54.521227+0000","last_undegraded":"2024-09-11T01:14:54.521227+0000","last_fullsized":"2024-09-11T01:14:54.521227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:19.905343+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10006528,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":86,"num_write_kb":344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'2444","reported_seq":2491,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.849098+0000","last_change":"2024-09-11T01:14:27.030470+0000","last_active":"2024-09-11T01:14:54.849098+0000","last_peered":"2024-09-11T01:14:54.849098+0000","last_clean":"2024-09-11T01:13:50.682082+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:54.849098+0000","last_undegraded":"2024-09-11T01:14:27.020390+0000","last_fullsized":"2024-09-11T01:14:54.849098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10002915,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":9769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'2459","reported_seq":2506,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.826963+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:54.826963+0000","last_peered":"2024-09-11T01:14:54.826963+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:54.826963+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:54.826963+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2459,"log_dups_size":0,"ondisk_log_size":2459,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10040745,"num_objects":2454,"num_object_clones":0,"num_object_copies":4908,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2456,"num_write_kb":9808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'2503","reported_seq":2550,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.840468+0000","last_change":"2024-09-11T01:14:27.030519+0000","last_active":"2024-09-11T01:14:54.840468+0000","last_peered":"2024-09-11T01:14:54.840468+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:54.840468+0000","last_undegraded":"2024-09-11T01:14:27.019380+0000","last_fullsized":"2024-09-11T01:14:54.840468+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10236870,"num_objects":2501,"num_object_clones":0,"num_object_copies":5002,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2501,"num_write_kb":9998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2501}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'2466","reported_seq":2512,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:54.858884+0000","last_change":"2024-09-11T01:14:27.030507+0000","last_active":"2024-09-11T01:14:54.858884+0000","last_peered":"2024-09-11T01:14:54.858884+0000","last_clean":"2024-09-11T01:14:00.501404+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:54.858884+0000","last_undegraded":"2024-09-11T01:14:27.019759+0000","last_fullsized":"2024-09-11T01:14:54.858884+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2466,"log_dups_size":0,"ondisk_log_size":2466,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10053516,"num_objects":2458,"num_object_clones":0,"num_object_copies":4916,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":159664997,"num_objects":39001,"num_object_clones":0,"num_object_copies":78002,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":39001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28474,"num_write_kb":113782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":319094784,"data_stored":318943038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39060,"ondisk_log_size":39060,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253160,"kb_used_data":153432,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94118680,"statfs":{"total":96636764160,"available":96377528320,"internally_reserved":0,"allocated":157114368,"data_stored":156912567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":102100654},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182414,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260304,"kb_used_data":156480,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94111536,"statfs":{"total":96636764160,"available":96370212864,"internally_reserved":0,"allocated":160235520,"data_stored":160007964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,3,13],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":160317440,"data_stored":160281310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158662656,"data_stored":158648204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:00.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 24 20 46918 46898 7.63242 4.67969 0.0160892 0.0102292 2024-09-11T01:15:01.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 25 20 48437 48417 7.56444 5.93359 0.014477 0.0103206 2024-09-11T01:15:02.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 26 20 49769 49749 7.47361 5.20312 0.0187393 0.0104467 2024-09-11T01:15:02.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:03.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 27 20 51048 51028 7.38183 4.99609 0.0122635 0.0105782 2024-09-11T01:15:03.443 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:03.445 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:03.465 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-11T01:15:02.415269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189012856,"num_objects":46168,"num_object_clones":0,"num_object_copies":92336,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":46168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35641,"num_write_kb":142444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46228,"ondisk_log_size":46228,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":592504,"kb_used_data":368152,"kb_used_omap":8,"kb_used_meta":224311,"kb_avail":188151176,"statfs":{"total":193273528320,"available":192666804224,"internally_reserved":0,"allocated":376987648,"data_stored":376558291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":229694808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,10,3,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59684497,"num_objects":14567,"num_object_clones":0,"num_object_copies":29134,"num_objects_missing_on_primary":-17,"num_objects_missing":-17,"num_objects_degraded":-33,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":14567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5913,"num_write_kb":23700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003221"},"pg_stats":[{"pgid":"2.f","version":"32'2904","reported_seq":2947,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.859462+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:59.859462+0000","last_peered":"2024-09-11T01:14:59.859462+0000","last_clean":"2024-09-11T01:14:59.859462+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:59.859462+0000","last_undegraded":"2024-09-11T01:14:59.859462+0000","last_fullsized":"2024-09-11T01:14:59.859462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11894784,"num_objects":2904,"num_object_clones":0,"num_object_copies":5808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":11616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'2884","reported_seq":2938,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.535833+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:14:59.535833+0000","last_peered":"2024-09-11T01:14:59.535833+0000","last_clean":"2024-09-11T01:14:59.535833+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:59.535833+0000","last_undegraded":"2024-09-11T01:14:59.535833+0000","last_fullsized":"2024-09-11T01:14:59.535833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":2884,"log_dups_size":0,"ondisk_log_size":2884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11789254,"num_objects":2880,"num_object_clones":0,"num_object_copies":5760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":982,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'2827","reported_seq":2877,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.498763+0000","last_change":"2024-09-11T01:14:50.131076+0000","last_active":"2024-09-11T01:14:59.498763+0000","last_peered":"2024-09-11T01:14:59.498763+0000","last_clean":"2024-09-11T01:14:59.498763+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:59.498763+0000","last_undegraded":"2024-09-11T01:14:59.498763+0000","last_fullsized":"2024-09-11T01:14:59.498763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":2827,"log_dups_size":0,"ondisk_log_size":2827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:54.197115+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11563008,"num_objects":2823,"num_object_clones":0,"num_object_copies":5646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'2878","reported_seq":2899,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.515010+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:59.515010+0000","last_peered":"2024-09-11T01:14:59.515010+0000","last_clean":"2024-09-11T01:14:59.515010+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:59.515010+0000","last_undegraded":"2024-09-11T01:14:59.515010+0000","last_fullsized":"2024-09-11T01:14:59.515010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2878,"log_dups_size":0,"ondisk_log_size":2878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11788288,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2878,"num_write_kb":11512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'2894","reported_seq":2949,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.861077+0000","last_change":"2024-09-11T01:14:59.781602+0000","last_active":"2024-09-11T01:14:59.861077+0000","last_peered":"2024-09-11T01:14:59.861077+0000","last_clean":"2024-09-11T01:14:59.861077+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:59.861077+0000","last_undegraded":"2024-09-11T01:14:59.861077+0000","last_fullsized":"2024-09-11T01:14:59.861077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2894,"log_dups_size":0,"ondisk_log_size":2894,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11826118,"num_objects":2889,"num_object_clones":0,"num_object_copies":5778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'2863","reported_seq":2911,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:59.871845+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:59.871845+0000","last_peered":"2024-09-11T01:14:59.871845+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:59.871845+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:59.871845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2863,"log_dups_size":0,"ondisk_log_size":2863,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11707334,"num_objects":2860,"num_object_clones":0,"num_object_copies":5720,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2861,"num_write_kb":11435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2859}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'3002","reported_seq":3059,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.848435+0000","last_change":"2024-09-11T01:14:59.772979+0000","last_active":"2024-09-11T01:14:59.848435+0000","last_peered":"2024-09-11T01:14:59.848435+0000","last_clean":"2024-09-11T01:14:59.848435+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:59.848435+0000","last_undegraded":"2024-09-11T01:14:59.848435+0000","last_fullsized":"2024-09-11T01:14:59.848435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3002,"log_dups_size":0,"ondisk_log_size":3002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12272582,"num_objects":2998,"num_object_clones":0,"num_object_copies":5996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'2893","reported_seq":2948,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.862150+0000","last_change":"2024-09-11T01:14:59.764253+0000","last_active":"2024-09-11T01:14:59.862150+0000","last_peered":"2024-09-11T01:14:59.862150+0000","last_clean":"2024-09-11T01:14:59.862150+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:59.862150+0000","last_undegraded":"2024-09-11T01:14:59.862150+0000","last_fullsized":"2024-09-11T01:14:59.862150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2893,"log_dups_size":0,"ondisk_log_size":2893,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11842019,"num_objects":2892,"num_object_clones":0,"num_object_copies":5784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'2866","reported_seq":2916,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.545533+0000","last_change":"2024-09-11T01:14:51.929341+0000","last_active":"2024-09-11T01:14:59.545533+0000","last_peered":"2024-09-11T01:14:59.545533+0000","last_clean":"2024-09-11T01:14:59.545533+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:59.545533+0000","last_undegraded":"2024-09-11T01:14:59.545533+0000","last_fullsized":"2024-09-11T01:14:59.545533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":2866,"log_dups_size":0,"ondisk_log_size":2866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:29.457390+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11710947,"num_objects":2860,"num_object_clones":0,"num_object_copies":5720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":2932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'2869","reported_seq":2910,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:59.847152+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:59.847152+0000","last_peered":"2024-09-11T01:14:59.847152+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:59.847152+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:59.847152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2869,"log_dups_size":0,"ondisk_log_size":2869,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2867,"num_write_kb":11448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'2804","reported_seq":2864,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.539374+0000","last_change":"2024-09-11T01:14:53.159504+0000","last_active":"2024-09-11T01:14:59.539374+0000","last_peered":"2024-09-11T01:14:59.539374+0000","last_clean":"2024-09-11T01:14:59.539374+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:59.539374+0000","last_undegraded":"2024-09-11T01:14:59.539374+0000","last_fullsized":"2024-09-11T01:14:59.539374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:11.511441+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11465187,"num_objects":2800,"num_object_clones":0,"num_object_copies":5600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":573,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'2889","reported_seq":2949,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.530360+0000","last_change":"2024-09-11T01:14:54.063365+0000","last_active":"2024-09-11T01:14:59.530360+0000","last_peered":"2024-09-11T01:14:59.530360+0000","last_clean":"2024-09-11T01:14:59.530360+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:59.530360+0000","last_undegraded":"2024-09-11T01:14:59.530360+0000","last_fullsized":"2024-09-11T01:14:59.530360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":2889,"log_dups_size":0,"ondisk_log_size":2889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:19.905343+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":526,"num_write_kb":2104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'2922","reported_seq":2978,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.860957+0000","last_change":"2024-09-11T01:14:59.786099+0000","last_active":"2024-09-11T01:14:59.860957+0000","last_peered":"2024-09-11T01:14:59.860957+0000","last_clean":"2024-09-11T01:14:59.860957+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:59.860957+0000","last_undegraded":"2024-09-11T01:14:59.860957+0000","last_fullsized":"2024-09-11T01:14:59.860957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2922,"log_dups_size":0,"ondisk_log_size":2922,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11960803,"num_objects":2921,"num_object_clones":0,"num_object_copies":5842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2921,"num_write_kb":11681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'2912","reported_seq":2959,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:59.821186+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:59.821186+0000","last_peered":"2024-09-11T01:14:59.821186+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:59.821186+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:59.821186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2912,"log_dups_size":0,"ondisk_log_size":2912,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11896233,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2909,"num_write_kb":11620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2907}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'2917","reported_seq":2969,"reported_epoch":32,"state":"active+recovering","last_fresh":"2024-09-11T01:14:59.861203+0000","last_change":"2024-09-11T01:14:59.825900+0000","last_active":"2024-09-11T01:14:59.861203+0000","last_peered":"2024-09-11T01:14:59.861203+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:59.861203+0000","last_undegraded":"2024-09-11T01:14:59.861203+0000","last_fullsized":"2024-09-11T01:14:59.861203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2917,"log_dups_size":0,"ondisk_log_size":2917,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932614,"num_objects":2915,"num_object_clones":0,"num_object_copies":5830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0,2","objects":2915}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'2901","reported_seq":2964,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.837980+0000","last_change":"2024-09-11T01:14:59.819140+0000","last_active":"2024-09-11T01:14:59.837980+0000","last_peered":"2024-09-11T01:14:59.837980+0000","last_clean":"2024-09-11T01:14:59.837980+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:59.837980+0000","last_undegraded":"2024-09-11T01:14:59.837980+0000","last_fullsized":"2024-09-11T01:14:59.837980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2901,"log_dups_size":0,"ondisk_log_size":2901,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11835276,"num_objects":2893,"num_object_clones":0,"num_object_copies":5786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2898,"num_write_kb":11563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189012837,"num_objects":46166,"num_object_clones":0,"num_object_copies":92332,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":46166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35639,"num_write_kb":142442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":378077184,"data_stored":377925438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46225,"ondisk_log_size":46225,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":295576,"kb_used_data":183368,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94076264,"statfs":{"total":96636764160,"available":96334094336,"internally_reserved":0,"allocated":187768832,"data_stored":187567031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114880174},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296928,"kb_used_data":184784,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94074912,"statfs":{"total":96636764160,"available":96332709888,"internally_reserved":0,"allocated":189218816,"data_stored":188991260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,1,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189370368,"data_stored":189334238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":188592128,"data_stored":188577676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:03.466 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:03.853 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:03.854 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:03.865 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-11T01:15:02.415269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":189012856,"num_objects":46168,"num_object_clones":0,"num_object_copies":92336,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":15,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":46168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35641,"num_write_kb":142444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46228,"ondisk_log_size":46228,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":592504,"kb_used_data":368152,"kb_used_omap":8,"kb_used_meta":224311,"kb_avail":188151176,"statfs":{"total":193273528320,"available":192666804224,"internally_reserved":0,"allocated":376987648,"data_stored":376558291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":229694808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,10,3,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":59684497,"num_objects":14567,"num_object_clones":0,"num_object_copies":29134,"num_objects_missing_on_primary":-17,"num_objects_missing":-17,"num_objects_degraded":-33,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":14567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5913,"num_write_kb":23700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003221"},"pg_stats":[{"pgid":"2.f","version":"32'2904","reported_seq":2947,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.859462+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:14:59.859462+0000","last_peered":"2024-09-11T01:14:59.859462+0000","last_clean":"2024-09-11T01:14:59.859462+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:14:59.859462+0000","last_undegraded":"2024-09-11T01:14:59.859462+0000","last_fullsized":"2024-09-11T01:14:59.859462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2904,"log_dups_size":0,"ondisk_log_size":2904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11894784,"num_objects":2904,"num_object_clones":0,"num_object_copies":5808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":11616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'2884","reported_seq":2938,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.535833+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:14:59.535833+0000","last_peered":"2024-09-11T01:14:59.535833+0000","last_clean":"2024-09-11T01:14:59.535833+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:14:59.535833+0000","last_undegraded":"2024-09-11T01:14:59.535833+0000","last_fullsized":"2024-09-11T01:14:59.535833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":2884,"log_dups_size":0,"ondisk_log_size":2884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11789254,"num_objects":2880,"num_object_clones":0,"num_object_copies":5760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":982,"num_write_kb":3928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'2827","reported_seq":2877,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.498763+0000","last_change":"2024-09-11T01:14:50.131076+0000","last_active":"2024-09-11T01:14:59.498763+0000","last_peered":"2024-09-11T01:14:59.498763+0000","last_clean":"2024-09-11T01:14:59.498763+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:14:59.498763+0000","last_undegraded":"2024-09-11T01:14:59.498763+0000","last_fullsized":"2024-09-11T01:14:59.498763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":2827,"log_dups_size":0,"ondisk_log_size":2827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:54.197115+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11563008,"num_objects":2823,"num_object_clones":0,"num_object_copies":5646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'2878","reported_seq":2899,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.515010+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:14:59.515010+0000","last_peered":"2024-09-11T01:14:59.515010+0000","last_clean":"2024-09-11T01:14:59.515010+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:14:59.515010+0000","last_undegraded":"2024-09-11T01:14:59.515010+0000","last_fullsized":"2024-09-11T01:14:59.515010+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2878,"log_dups_size":0,"ondisk_log_size":2878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11788288,"num_objects":2878,"num_object_clones":0,"num_object_copies":5756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2878,"num_write_kb":11512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'2894","reported_seq":2949,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.861077+0000","last_change":"2024-09-11T01:14:59.781602+0000","last_active":"2024-09-11T01:14:59.861077+0000","last_peered":"2024-09-11T01:14:59.861077+0000","last_clean":"2024-09-11T01:14:59.861077+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:14:59.861077+0000","last_undegraded":"2024-09-11T01:14:59.861077+0000","last_fullsized":"2024-09-11T01:14:59.861077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2894,"log_dups_size":0,"ondisk_log_size":2894,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11826118,"num_objects":2889,"num_object_clones":0,"num_object_copies":5778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'2863","reported_seq":2911,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:59.871845+0000","last_change":"2024-09-11T01:14:27.030688+0000","last_active":"2024-09-11T01:14:59.871845+0000","last_peered":"2024-09-11T01:14:59.871845+0000","last_clean":"2024-09-11T01:14:00.814126+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:14:59.871845+0000","last_undegraded":"2024-09-11T01:14:27.020206+0000","last_fullsized":"2024-09-11T01:14:59.871845+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2863,"log_dups_size":0,"ondisk_log_size":2863,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11707334,"num_objects":2860,"num_object_clones":0,"num_object_copies":5720,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2861,"num_write_kb":11435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2859}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'3002","reported_seq":3059,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.848435+0000","last_change":"2024-09-11T01:14:59.772979+0000","last_active":"2024-09-11T01:14:59.848435+0000","last_peered":"2024-09-11T01:14:59.848435+0000","last_clean":"2024-09-11T01:14:59.848435+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:14:59.848435+0000","last_undegraded":"2024-09-11T01:14:59.848435+0000","last_fullsized":"2024-09-11T01:14:59.848435+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3002,"log_dups_size":0,"ondisk_log_size":3002,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12272582,"num_objects":2998,"num_object_clones":0,"num_object_copies":5996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":11988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'2893","reported_seq":2948,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.862150+0000","last_change":"2024-09-11T01:14:59.764253+0000","last_active":"2024-09-11T01:14:59.862150+0000","last_peered":"2024-09-11T01:14:59.862150+0000","last_clean":"2024-09-11T01:14:59.862150+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:14:59.862150+0000","last_undegraded":"2024-09-11T01:14:59.862150+0000","last_fullsized":"2024-09-11T01:14:59.862150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2893,"log_dups_size":0,"ondisk_log_size":2893,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11842019,"num_objects":2892,"num_object_clones":0,"num_object_copies":5784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'2866","reported_seq":2916,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.545533+0000","last_change":"2024-09-11T01:14:51.929341+0000","last_active":"2024-09-11T01:14:59.545533+0000","last_peered":"2024-09-11T01:14:59.545533+0000","last_clean":"2024-09-11T01:14:59.545533+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:14:59.545533+0000","last_undegraded":"2024-09-11T01:14:59.545533+0000","last_fullsized":"2024-09-11T01:14:59.545533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":2866,"log_dups_size":0,"ondisk_log_size":2866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:29.457390+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11710947,"num_objects":2860,"num_object_clones":0,"num_object_copies":5720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2860,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":733,"num_write_kb":2932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'2869","reported_seq":2910,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:59.847152+0000","last_change":"2024-09-11T01:14:27.031003+0000","last_active":"2024-09-11T01:14:59.847152+0000","last_peered":"2024-09-11T01:14:59.847152+0000","last_clean":"2024-09-11T01:14:00.865944+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:14:59.847152+0000","last_undegraded":"2024-09-11T01:14:27.020573+0000","last_fullsized":"2024-09-11T01:14:59.847152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2869,"log_dups_size":0,"ondisk_log_size":2869,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2867,"num_write_kb":11448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:27.028485+0000","last_change":"2024-09-11T01:14:27.028654+0000","last_active":"2024-09-11T01:14:27.028485+0000","last_peered":"2024-09-11T01:14:27.028485+0000","last_clean":"2024-09-11T01:14:11.846547+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:14:27.028485+0000","last_undegraded":"2024-09-11T01:14:27.019624+0000","last_fullsized":"2024-09-11T01:14:27.028485+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'2804","reported_seq":2864,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.539374+0000","last_change":"2024-09-11T01:14:53.159504+0000","last_active":"2024-09-11T01:14:59.539374+0000","last_peered":"2024-09-11T01:14:59.539374+0000","last_clean":"2024-09-11T01:14:59.539374+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:14:59.539374+0000","last_undegraded":"2024-09-11T01:14:59.539374+0000","last_fullsized":"2024-09-11T01:14:59.539374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:11.511441+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11465187,"num_objects":2800,"num_object_clones":0,"num_object_copies":5600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":573,"num_write_kb":2292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'2889","reported_seq":2949,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.530360+0000","last_change":"2024-09-11T01:14:54.063365+0000","last_active":"2024-09-11T01:14:59.530360+0000","last_peered":"2024-09-11T01:14:59.530360+0000","last_clean":"2024-09-11T01:14:59.530360+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:14:59.530360+0000","last_undegraded":"2024-09-11T01:14:59.530360+0000","last_fullsized":"2024-09-11T01:14:59.530360+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":2889,"log_dups_size":0,"ondisk_log_size":2889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:19.905343+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":526,"num_write_kb":2104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'2922","reported_seq":2978,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.860957+0000","last_change":"2024-09-11T01:14:59.786099+0000","last_active":"2024-09-11T01:14:59.860957+0000","last_peered":"2024-09-11T01:14:59.860957+0000","last_clean":"2024-09-11T01:14:59.860957+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:14:59.860957+0000","last_undegraded":"2024-09-11T01:14:59.860957+0000","last_fullsized":"2024-09-11T01:14:59.860957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2922,"log_dups_size":0,"ondisk_log_size":2922,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11960803,"num_objects":2921,"num_object_clones":0,"num_object_copies":5842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2921,"num_write_kb":11681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'2912","reported_seq":2959,"reported_epoch":32,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:14:59.821186+0000","last_change":"2024-09-11T01:14:27.030783+0000","last_active":"2024-09-11T01:14:59.821186+0000","last_peered":"2024-09-11T01:14:59.821186+0000","last_clean":"2024-09-11T01:14:00.762159+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:14:59.821186+0000","last_undegraded":"2024-09-11T01:14:27.020137+0000","last_fullsized":"2024-09-11T01:14:59.821186+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2912,"log_dups_size":0,"ondisk_log_size":2912,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11896233,"num_objects":2907,"num_object_clones":0,"num_object_copies":5814,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2909,"num_write_kb":11620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2907}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'2917","reported_seq":2969,"reported_epoch":32,"state":"active+recovering","last_fresh":"2024-09-11T01:14:59.861203+0000","last_change":"2024-09-11T01:14:59.825900+0000","last_active":"2024-09-11T01:14:59.861203+0000","last_peered":"2024-09-11T01:14:59.861203+0000","last_clean":"2024-09-11T01:13:50.682009+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:14:59.861203+0000","last_undegraded":"2024-09-11T01:14:59.861203+0000","last_fullsized":"2024-09-11T01:14:59.861203+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2917,"log_dups_size":0,"ondisk_log_size":2917,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11932614,"num_objects":2915,"num_object_clones":0,"num_object_copies":5830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2915,"num_write_kb":11654,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0,2","objects":2915}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'2901","reported_seq":2964,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.837980+0000","last_change":"2024-09-11T01:14:59.819140+0000","last_active":"2024-09-11T01:14:59.837980+0000","last_peered":"2024-09-11T01:14:59.837980+0000","last_clean":"2024-09-11T01:14:59.837980+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:14:59.837980+0000","last_undegraded":"2024-09-11T01:14:59.837980+0000","last_fullsized":"2024-09-11T01:14:59.837980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":2901,"log_dups_size":0,"ondisk_log_size":2901,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11835276,"num_objects":2893,"num_object_clones":0,"num_object_copies":5786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2898,"num_write_kb":11563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":189012837,"num_objects":46166,"num_object_clones":0,"num_object_copies":92332,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":14,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":46166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35639,"num_write_kb":142442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":2415,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":378077184,"data_stored":377925438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":46225,"ondisk_log_size":46225,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":295576,"kb_used_data":183368,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94076264,"statfs":{"total":96636764160,"available":96334094336,"internally_reserved":0,"allocated":187768832,"data_stored":187567031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":114880174},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,2,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296928,"kb_used_data":184784,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94074912,"statfs":{"total":96636764160,"available":96332709888,"internally_reserved":0,"allocated":189218816,"data_stored":188991260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,1,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":189370368,"data_stored":189334238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":188592128,"data_stored":188577676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:04.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 28 20 52048 52028 7.25769 3.90625 0.0200838 0.010757 2024-09-11T01:15:05.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 29 20 53387 53367 7.18778 5.23047 0.0206527 0.0108636 2024-09-11T01:15:06.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 30 20 54726 54706 7.12252 5.23047 0.0165384 0.0109629 2024-09-11T01:15:06.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:07.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 31 20 55906 55886 7.04144 4.60938 0.00771828 0.0110867 2024-09-11T01:15:07.420 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:07.420 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:07.444 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-09-11T01:15:06.416028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":215734194,"num_objects":52690,"num_object_clones":0,"num_object_copies":105380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32674,"num_write_kb":130607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":1468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52752,"ondisk_log_size":52752,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":667968,"kb_used_data":420000,"kb_used_omap":8,"kb_used_meta":247927,"kb_avail":188075712,"statfs":{"total":193273528320,"available":192589529088,"internally_reserved":0,"allocated":430080000,"data_stored":429624888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8872,"internal_metadata":253877592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,5,13,14,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":86405835,"num_objects":21089,"num_object_clones":0,"num_object_copies":42178,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-48,"num_objects_misplaced":0,"num_objects_unfound":-21,"num_objects_dirty":21089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2946,"num_write_kb":11863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1468,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002596"},"pg_stats":[{"pgid":"2.f","version":"32'3299","reported_seq":3341,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.898954+0000","last_change":"2024-09-11T01:14:27.028998+0000","last_active":"2024-09-11T01:15:04.898954+0000","last_peered":"2024-09-11T01:15:04.898954+0000","last_clean":"2024-09-11T01:15:04.898954+0000","last_became_active":"2024-09-11T01:14:27.028746+0000","last_became_peered":"2024-09-11T01:14:27.028746+0000","last_unstale":"2024-09-11T01:15:04.898954+0000","last_undegraded":"2024-09-11T01:15:04.898954+0000","last_fullsized":"2024-09-11T01:15:04.898954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3299,"log_dups_size":0,"ondisk_log_size":3299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:20:07.431319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13512704,"num_objects":3299,"num_object_clones":0,"num_object_copies":6598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3299,"num_write_kb":13196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'3258","reported_seq":3312,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.541135+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:04.541135+0000","last_peered":"2024-09-11T01:15:04.541135+0000","last_clean":"2024-09-11T01:15:04.541135+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:04.541135+0000","last_undegraded":"2024-09-11T01:15:04.541135+0000","last_fullsized":"2024-09-11T01:15:04.541135+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":3258,"log_dups_size":0,"ondisk_log_size":3258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13321158,"num_objects":3254,"num_object_clones":0,"num_object_copies":6508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1356,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'3219","reported_seq":3269,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.536822+0000","last_change":"2024-09-11T01:14:50.131076+0000","last_active":"2024-09-11T01:15:04.536822+0000","last_peered":"2024-09-11T01:15:04.536822+0000","last_clean":"2024-09-11T01:15:04.536822+0000","last_became_active":"2024-09-11T01:14:27.027959+0000","last_became_peered":"2024-09-11T01:14:27.027959+0000","last_unstale":"2024-09-11T01:15:04.536822+0000","last_undegraded":"2024-09-11T01:15:04.536822+0000","last_fullsized":"2024-09-11T01:15:04.536822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":3219,"log_dups_size":0,"ondisk_log_size":3219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:54.197115+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13168640,"num_objects":3215,"num_object_clones":0,"num_object_copies":6430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1280,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'3257","reported_seq":3278,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.532568+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:04.532568+0000","last_peered":"2024-09-11T01:15:04.532568+0000","last_clean":"2024-09-11T01:15:04.532568+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:04.532568+0000","last_undegraded":"2024-09-11T01:15:04.532568+0000","last_fullsized":"2024-09-11T01:15:04.532568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3257,"log_dups_size":0,"ondisk_log_size":3257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3257,"num_write_kb":13028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'3304","reported_seq":3400,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.870497+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:04.870497+0000","last_peered":"2024-09-11T01:15:04.870497+0000","last_clean":"2024-09-11T01:15:04.870497+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:04.870497+0000","last_undegraded":"2024-09-11T01:15:04.870497+0000","last_fullsized":"2024-09-11T01:15:04.870497+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":3304,"log_dups_size":0,"ondisk_log_size":3304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13504995,"num_objects":3298,"num_object_clones":0,"num_object_copies":6596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102,"num_write_kb":408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'3276","reported_seq":3334,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.858839+0000","last_change":"2024-09-11T01:14:59.946997+0000","last_active":"2024-09-11T01:15:04.858839+0000","last_peered":"2024-09-11T01:15:04.858839+0000","last_clean":"2024-09-11T01:15:04.858839+0000","last_became_active":"2024-09-11T01:14:27.030325+0000","last_became_peered":"2024-09-11T01:14:27.030325+0000","last_unstale":"2024-09-11T01:15:04.858839+0000","last_undegraded":"2024-09-11T01:15:04.858839+0000","last_fullsized":"2024-09-11T01:15:04.858839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3276,"log_dups_size":0,"ondisk_log_size":3276,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13398982,"num_objects":3273,"num_object_clones":0,"num_object_copies":6546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3274,"num_write_kb":13087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'3423","reported_seq":3523,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.858584+0000","last_change":"2024-09-11T01:15:03.099566+0000","last_active":"2024-09-11T01:15:04.858584+0000","last_peered":"2024-09-11T01:15:04.858584+0000","last_clean":"2024-09-11T01:15:04.858584+0000","last_became_active":"2024-09-11T01:14:27.029153+0000","last_became_peered":"2024-09-11T01:14:27.029153+0000","last_unstale":"2024-09-11T01:15:04.858584+0000","last_undegraded":"2024-09-11T01:15:04.858584+0000","last_fullsized":"2024-09-11T01:15:04.858584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":3423,"log_dups_size":0,"ondisk_log_size":3423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:09.533129+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13996998,"num_objects":3419,"num_object_clones":0,"num_object_copies":6838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3419,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":176,"num_write_kb":704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'3320","reported_seq":3414,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.868208+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:04.868208+0000","last_peered":"2024-09-11T01:15:04.868208+0000","last_clean":"2024-09-11T01:15:04.868208+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:04.868208+0000","last_undegraded":"2024-09-11T01:15:04.868208+0000","last_fullsized":"2024-09-11T01:15:04.868208+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":3320,"log_dups_size":0,"ondisk_log_size":3320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13590528,"num_objects":3318,"num_object_clones":0,"num_object_copies":6636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":273,"num_write_kb":1092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030562+0000","last_change":"2024-09-11T01:14:27.030476+0000","last_active":"2024-09-11T01:14:27.030562+0000","last_peered":"2024-09-11T01:14:27.030562+0000","last_clean":"2024-09-11T01:14:27.030562+0000","last_became_active":"2024-09-11T01:14:27.030095+0000","last_became_peered":"2024-09-11T01:14:27.030095+0000","last_unstale":"2024-09-11T01:14:27.030562+0000","last_undegraded":"2024-09-11T01:14:27.030562+0000","last_fullsized":"2024-09-11T01:14:27.030562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:38.876574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'3309","reported_seq":3358,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.550610+0000","last_change":"2024-09-11T01:14:51.929341+0000","last_active":"2024-09-11T01:15:04.550610+0000","last_peered":"2024-09-11T01:15:04.550610+0000","last_clean":"2024-09-11T01:15:04.550610+0000","last_became_active":"2024-09-11T01:14:27.027502+0000","last_became_peered":"2024-09-11T01:14:27.027502+0000","last_unstale":"2024-09-11T01:15:04.550610+0000","last_undegraded":"2024-09-11T01:15:04.550610+0000","last_fullsized":"2024-09-11T01:15:04.550610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":3309,"log_dups_size":0,"ondisk_log_size":3309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:29.457390+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13525475,"num_objects":3303,"num_object_clones":0,"num_object_copies":6606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1176,"num_write_kb":4704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.026954+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:14:27.026954+0000","last_peered":"2024-09-11T01:14:27.026954+0000","last_clean":"2024-09-11T01:14:27.026954+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:14:27.026954+0000","last_undegraded":"2024-09-11T01:14:27.026954+0000","last_fullsized":"2024-09-11T01:14:27.026954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'3278","reported_seq":3331,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.888396+0000","last_change":"2024-09-11T01:15:00.064968+0000","last_active":"2024-09-11T01:15:04.888396+0000","last_peered":"2024-09-11T01:15:04.888396+0000","last_clean":"2024-09-11T01:15:04.888396+0000","last_became_active":"2024-09-11T01:14:27.030754+0000","last_became_peered":"2024-09-11T01:14:27.030754+0000","last_unstale":"2024-09-11T01:15:04.888396+0000","last_undegraded":"2024-09-11T01:15:04.888396+0000","last_fullsized":"2024-09-11T01:15:04.888396+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3278,"log_dups_size":0,"ondisk_log_size":3278,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13394886,"num_objects":3272,"num_object_clones":0,"num_object_copies":6544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3276,"num_write_kb":13084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":63,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:00.093007+0000","last_change":"2024-09-11T01:15:00.092902+0000","last_active":"2024-09-11T01:15:00.093007+0000","last_peered":"2024-09-11T01:15:00.093007+0000","last_clean":"2024-09-11T01:15:00.093007+0000","last_became_active":"2024-09-11T01:14:27.028385+0000","last_became_peered":"2024-09-11T01:14:27.028385+0000","last_unstale":"2024-09-11T01:15:00.093007+0000","last_undegraded":"2024-09-11T01:15:00.093007+0000","last_fullsized":"2024-09-11T01:15:00.093007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:41:52.146325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'3173","reported_seq":3234,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.463701+0000","last_change":"2024-09-11T01:14:53.159504+0000","last_active":"2024-09-11T01:15:04.463701+0000","last_peered":"2024-09-11T01:15:04.463701+0000","last_clean":"2024-09-11T01:15:04.463701+0000","last_became_active":"2024-09-11T01:14:27.027243+0000","last_became_peered":"2024-09-11T01:14:27.027243+0000","last_unstale":"2024-09-11T01:15:04.463701+0000","last_undegraded":"2024-09-11T01:15:04.463701+0000","last_fullsized":"2024-09-11T01:15:04.463701+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":3173,"log_dups_size":0,"ondisk_log_size":3173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:48:11.511441+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976611,"num_objects":3169,"num_object_clones":0,"num_object_copies":6338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":942,"num_write_kb":3768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027109+0000","last_change":"2024-09-11T01:14:27.027033+0000","last_active":"2024-09-11T01:14:27.027109+0000","last_peered":"2024-09-11T01:14:27.027109+0000","last_clean":"2024-09-11T01:14:27.027109+0000","last_became_active":"2024-09-11T01:14:27.026728+0000","last_became_peered":"2024-09-11T01:14:27.026728+0000","last_unstale":"2024-09-11T01:14:27.027109+0000","last_undegraded":"2024-09-11T01:14:27.027109+0000","last_fullsized":"2024-09-11T01:14:27.027109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:30:50.465981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'3332","reported_seq":3393,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.519512+0000","last_change":"2024-09-11T01:14:54.063365+0000","last_active":"2024-09-11T01:15:04.519512+0000","last_peered":"2024-09-11T01:15:04.519512+0000","last_clean":"2024-09-11T01:15:04.519512+0000","last_became_active":"2024-09-11T01:14:27.026773+0000","last_became_peered":"2024-09-11T01:14:27.026773+0000","last_unstale":"2024-09-11T01:15:04.519512+0000","last_undegraded":"2024-09-11T01:15:04.519512+0000","last_fullsized":"2024-09-11T01:15:04.519512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":3332,"log_dups_size":0,"ondisk_log_size":3332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:19.905343+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13623296,"num_objects":3326,"num_object_clones":0,"num_object_copies":6652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":969,"num_write_kb":3876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'3357","reported_seq":3445,"reported_epoch":32,"state":"active+clean+scrubbing","last_fresh":"2024-09-11T01:15:04.897473+0000","last_change":"2024-09-11T01:15:03.838934+0000","last_active":"2024-09-11T01:15:04.897473+0000","last_peered":"2024-09-11T01:15:04.897473+0000","last_clean":"2024-09-11T01:15:04.897473+0000","last_became_active":"2024-09-11T01:14:27.030216+0000","last_became_peered":"2024-09-11T01:14:27.030216+0000","last_unstale":"2024-09-11T01:15:04.897473+0000","last_undegraded":"2024-09-11T01:15:04.897473+0000","last_fullsized":"2024-09-11T01:15:04.897473+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":2799,"log_size":3357,"log_dups_size":0,"ondisk_log_size":3357,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 1s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13742563,"num_objects":3356,"num_object_clones":0,"num_object_copies":6712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3356,"num_write_kb":13421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.030216+0000","last_change":"2024-09-11T01:14:27.030124+0000","last_active":"2024-09-11T01:14:27.030216+0000","last_peered":"2024-09-11T01:14:27.030216+0000","last_clean":"2024-09-11T01:14:27.030216+0000","last_became_active":"2024-09-11T01:14:27.029773+0000","last_became_peered":"2024-09-11T01:14:27.029773+0000","last_unstale":"2024-09-11T01:14:27.030216+0000","last_undegraded":"2024-09-11T01:14:27.030216+0000","last_fullsized":"2024-09-11T01:14:27.030216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:52.961362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'3312","reported_seq":3373,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.890636+0000","last_change":"2024-09-11T01:15:00.016887+0000","last_active":"2024-09-11T01:15:04.890636+0000","last_peered":"2024-09-11T01:15:04.890636+0000","last_clean":"2024-09-11T01:15:04.890636+0000","last_became_active":"2024-09-11T01:14:27.029859+0000","last_became_peered":"2024-09-11T01:14:27.029859+0000","last_unstale":"2024-09-11T01:15:04.890636+0000","last_undegraded":"2024-09-11T01:15:04.890636+0000","last_fullsized":"2024-09-11T01:15:04.890636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3312,"log_dups_size":0,"ondisk_log_size":3312,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13534633,"num_objects":3307,"num_object_clones":0,"num_object_copies":6614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3309,"num_write_kb":13220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":36,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028628+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:14:27.028628+0000","last_peered":"2024-09-11T01:14:27.028628+0000","last_clean":"2024-09-11T01:14:27.028628+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:14:27.028628+0000","last_undegraded":"2024-09-11T01:14:27.028628+0000","last_fullsized":"2024-09-11T01:14:27.028628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'3334","reported_seq":3392,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.898207+0000","last_change":"2024-09-11T01:14:59.881314+0000","last_active":"2024-09-11T01:15:04.898207+0000","last_peered":"2024-09-11T01:15:04.898207+0000","last_clean":"2024-09-11T01:15:04.898207+0000","last_became_active":"2024-09-11T01:14:27.029981+0000","last_became_peered":"2024-09-11T01:14:27.029981+0000","last_unstale":"2024-09-11T01:15:04.898207+0000","last_undegraded":"2024-09-11T01:15:04.898207+0000","last_fullsized":"2024-09-11T01:15:04.898207+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3334,"log_dups_size":0,"ondisk_log_size":3334,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13640646,"num_objects":3332,"num_object_clones":0,"num_object_copies":6664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3332,"num_write_kb":13322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":67,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.028052+0000","last_change":"2024-09-11T01:14:27.027959+0000","last_active":"2024-09-11T01:14:27.028052+0000","last_peered":"2024-09-11T01:14:27.028052+0000","last_clean":"2024-09-11T01:14:27.028052+0000","last_became_active":"2024-09-11T01:14:27.027669+0000","last_became_peered":"2024-09-11T01:14:27.027669+0000","last_unstale":"2024-09-11T01:14:27.028052+0000","last_undegraded":"2024-09-11T01:14:27.028052+0000","last_fullsized":"2024-09-11T01:14:27.028052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:40.426545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T01:14:27.027663+0000","last_change":"2024-09-11T01:14:27.027568+0000","last_active":"2024-09-11T01:14:27.027663+0000","last_peered":"2024-09-11T01:14:27.027663+0000","last_clean":"2024-09-11T01:14:27.027663+0000","last_became_active":"2024-09-11T01:14:27.027169+0000","last_became_peered":"2024-09-11T01:14:27.027169+0000","last_unstale":"2024-09-11T01:14:27.027663+0000","last_undegraded":"2024-09-11T01:14:27.027663+0000","last_fullsized":"2024-09-11T01:14:27.027663+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:35.263360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'3298","reported_seq":3365,"reported_epoch":32,"state":"active+clean+scrubbing","last_fresh":"2024-09-11T01:15:04.898235+0000","last_change":"2024-09-11T01:15:04.888218+0000","last_active":"2024-09-11T01:15:04.898235+0000","last_peered":"2024-09-11T01:15:04.898235+0000","last_clean":"2024-09-11T01:15:04.898235+0000","last_became_active":"2024-09-11T01:14:27.030237+0000","last_became_peered":"2024-09-11T01:14:27.030237+0000","last_unstale":"2024-09-11T01:15:04.898235+0000","last_undegraded":"2024-09-11T01:15:04.898235+0000","last_fullsized":"2024-09-11T01:15:04.898235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":3298,"log_dups_size":0,"ondisk_log_size":3298,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13461388,"num_objects":3290,"num_object_clones":0,"num_object_copies":6580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3295,"num_write_kb":13151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":215734175,"num_objects":52688,"num_object_clones":0,"num_object_copies":105376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32672,"num_write_kb":130605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":431153152,"data_stored":430979728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52749,"ondisk_log_size":52749,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":330316,"kb_used_data":208380,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94041524,"statfs":{"total":96636764160,"available":96298520576,"internally_reserved":0,"allocated":213381120,"data_stored":213153564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":124841646},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,3,5,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337652,"kb_used_data":211620,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94034188,"statfs":{"total":96636764160,"available":96291008512,"internally_reserved":0,"allocated":216698880,"data_stored":216471324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":129035946},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":216469504,"data_stored":216433374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":214568960,"data_stored":214532830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:07.446 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-11T01:15:07.446 INFO:tasks.util.rados:rados -p unfound_pool get new_1 - 2024-09-11T01:15:07.446 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_1 - 2024-09-11T01:15:07.554 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_1: (2) No such file or directory 2024-09-11T01:15:07.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:07.561 INFO:tasks.util.rados:rados -p unfound_pool get existed_1 - 2024-09-11T01:15:07.561 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_1 - 2024-09-11T01:15:07.659 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_1: (2) No such file or directory 2024-09-11T01:15:07.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:07.667 INFO:tasks.util.rados:rados -p unfound_pool get existing_1 - 2024-09-11T01:15:07.667 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_1 - 2024-09-11T01:15:07.779 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:07.780 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:07.781 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:07.781 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:07.781 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:07.781 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:07.786 INFO:tasks.util.rados:rados -p unfound_pool get new_2 - 2024-09-11T01:15:07.786 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_2 - 2024-09-11T01:15:07.890 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_2: (2) No such file or directory 2024-09-11T01:15:07.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:07.897 INFO:tasks.util.rados:rados -p unfound_pool get existed_2 - 2024-09-11T01:15:07.897 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_2 - 2024-09-11T01:15:08.005 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_2: (2) No such file or directory 2024-09-11T01:15:08.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:08.015 INFO:tasks.util.rados:rados -p unfound_pool get existing_2 - 2024-09-11T01:15:08.015 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_2 - 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:08.127 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.128 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:08.128 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:08.128 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.128 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:08.128 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:08.140 INFO:tasks.util.rados:rados -p unfound_pool get new_3 - 2024-09-11T01:15:08.140 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_3 - 2024-09-11T01:15:08.237 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_3: (2) No such file or directory 2024-09-11T01:15:08.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:08.257 INFO:tasks.util.rados:rados -p unfound_pool get existed_3 - 2024-09-11T01:15:08.258 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_3 - 2024-09-11T01:15:08.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 32 20 57026 57006 6.9581 4.375 0.02179 0.0112218 2024-09-11T01:15:08.353 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_3: (2) No such file or directory 2024-09-11T01:15:08.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:08.360 INFO:tasks.util.rados:rados -p unfound_pool get existing_3 - 2024-09-11T01:15:08.361 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_3 - 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.468 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:08.469 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:08.469 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.469 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:08.469 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:08.473 INFO:tasks.util.rados:rados -p unfound_pool get new_4 - 2024-09-11T01:15:08.473 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_4 - 2024-09-11T01:15:08.582 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_4: (2) No such file or directory 2024-09-11T01:15:08.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:08.595 INFO:tasks.util.rados:rados -p unfound_pool get existed_4 - 2024-09-11T01:15:08.595 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_4 - 2024-09-11T01:15:08.701 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_4: (2) No such file or directory 2024-09-11T01:15:08.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:08.713 INFO:tasks.util.rados:rados -p unfound_pool get existing_4 - 2024-09-11T01:15:08.713 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_4 - 2024-09-11T01:15:08.802 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:08.802 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:08.803 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:08.809 INFO:tasks.util.rados:rados -p unfound_pool get new_5 - 2024-09-11T01:15:08.809 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_5 - 2024-09-11T01:15:08.896 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_5: (2) No such file or directory 2024-09-11T01:15:08.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:08.901 INFO:tasks.util.rados:rados -p unfound_pool get existed_5 - 2024-09-11T01:15:08.901 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_5 - 2024-09-11T01:15:09.017 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_5: (2) No such file or directory 2024-09-11T01:15:09.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:09.039 INFO:tasks.util.rados:rados -p unfound_pool get existing_5 - 2024-09-11T01:15:09.039 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_5 - 2024-09-11T01:15:09.148 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:09.149 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:09.157 INFO:tasks.util.rados:rados -p unfound_pool get new_6 - 2024-09-11T01:15:09.157 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_6 - 2024-09-11T01:15:09.264 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_6: (2) No such file or directory 2024-09-11T01:15:09.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:09.275 INFO:tasks.util.rados:rados -p unfound_pool get existed_6 - 2024-09-11T01:15:09.276 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_6 - 2024-09-11T01:15:09.314 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 33 20 58116 58096 6.87627 4.25781 0.0160104 0.011354 2024-09-11T01:15:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_6: (2) No such file or directory 2024-09-11T01:15:09.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:09.392 INFO:tasks.util.rados:rados -p unfound_pool get existing_6 - 2024-09-11T01:15:09.392 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_6 - 2024-09-11T01:15:09.529 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:09.529 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.529 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:09.530 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:09.532 INFO:tasks.util.rados:rados -p unfound_pool get new_7 - 2024-09-11T01:15:09.532 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_7 - 2024-09-11T01:15:09.636 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_7: (2) No such file or directory 2024-09-11T01:15:09.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:09.643 INFO:tasks.util.rados:rados -p unfound_pool get existed_7 - 2024-09-11T01:15:09.643 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_7 - 2024-09-11T01:15:09.750 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_7: (2) No such file or directory 2024-09-11T01:15:09.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:09.765 INFO:tasks.util.rados:rados -p unfound_pool get existing_7 - 2024-09-11T01:15:09.766 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_7 - 2024-09-11T01:15:09.871 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:09.871 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.871 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:09.871 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:09.871 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:09.872 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:09.886 INFO:tasks.util.rados:rados -p unfound_pool get new_8 - 2024-09-11T01:15:09.886 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_8 - 2024-09-11T01:15:09.996 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_8: (2) No such file or directory 2024-09-11T01:15:10.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:10.006 INFO:tasks.util.rados:rados -p unfound_pool get existed_8 - 2024-09-11T01:15:10.007 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_8 - 2024-09-11T01:15:10.100 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_8: (2) No such file or directory 2024-09-11T01:15:10.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:10.106 INFO:tasks.util.rados:rados -p unfound_pool get existing_8 - 2024-09-11T01:15:10.106 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_8 - 2024-09-11T01:15:10.199 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:10.199 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.199 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:10.199 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:10.199 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.199 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:10.200 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:10.211 INFO:tasks.util.rados:rados -p unfound_pool get new_9 - 2024-09-11T01:15:10.211 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_9 - 2024-09-11T01:15:10.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 34 20 59187 59167 6.79706 4.18359 0.0119676 0.0114876 2024-09-11T01:15:10.316 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/new_9: (2) No such file or directory 2024-09-11T01:15:10.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:10.324 INFO:tasks.util.rados:rados -p unfound_pool get existed_9 - 2024-09-11T01:15:10.325 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_9 - 2024-09-11T01:15:10.422 INFO:teuthology.orchestra.run.smithi039.stderr:error getting unfound_pool/existed_9: (2) No such file or directory 2024-09-11T01:15:10.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:15:10.432 INFO:tasks.util.rados:rados -p unfound_pool get existing_9 - 2024-09-11T01:15:10.433 DEBUG:teuthology.orchestra.run.smithi039:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_9 - 2024-09-11T01:15:10.518 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:10.518 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.518 INFO:teuthology.orchestra.run.smithi039.stdout:# /etc/fstab 2024-09-11T01:15:10.518 INFO:teuthology.orchestra.run.smithi039.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-11T01:15:10.518 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.518 INFO:teuthology.orchestra.run.smithi039.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:# units generated from this file. 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:# 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-11T01:15:10.519 INFO:teuthology.orchestra.run.smithi039.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-11T01:15:10.525 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-11T01:15:11.097 INFO:tasks.ceph.mon.a.smithi039.stderr:2024-09-11T01:15:11.095+0000 7f7c4de44640 -1 mon.a@0(leader).osd e32 definitely_dead 0 2024-09-11T01:15:11.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 35 20 60421 60401 6.74057 4.82031 0.0133641 0.0115847 2024-09-11T01:15:11.898 INFO:teuthology.orchestra.run.smithi039.stderr:marked in osd.1. 2024-09-11T01:15:11.926 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T01:15:11.926 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T01:15:11.928 INFO:tasks.ceph.osd.1:Started 2024-09-11T01:15:11.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T01:15:12.030 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-11T01:15:12.030 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-11T01:15:12.030 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-11T01:15:12.030 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-11T01:15:12.030 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-11T01:15:12.030 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-11T01:15:12.031 INFO:teuthology.orchestra.run.smithi039.stderr:config diff 2024-09-11T01:15:12.031 INFO:teuthology.orchestra.run.smithi039.stderr:config diff get 2024-09-11T01:15:12.031 INFO:teuthology.orchestra.run.smithi039.stderr:config get 2024-09-11T01:15:12.031 INFO:teuthology.orchestra.run.smithi039.stderr:config help [] 2024-09-11T01:15:12.031 INFO:teuthology.orchestra.run.smithi039.stderr:config set ... 2024-09-11T01:15:12.031 INFO:teuthology.orchestra.run.smithi039.stderr:admin_socket: invalid command 2024-09-11T01:15:12.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:15:12.032 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-11T01:15:12.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 36 20 61234 61214 6.64154 3.17578 0.00275374 0.0116248 2024-09-11T01:15:12.859 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:15:12.858+0000 7f058360c640 -1 Falling back to public interface 2024-09-11T01:15:13.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 37 20 61234 61214 6.46203 0 - 0.0116248 2024-09-11T01:15:14.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 38 20 61714 61694 6.34132 0.9375 0.0138 0.0123148 2024-09-11T01:15:15.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 39 20 63024 63004 6.30992 5.11719 0.0176366 0.0123756 2024-09-11T01:15:16.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:15:16.314729+0000 min lat: 0.00221854 max lat: 2.08201 avg lat: 0.0124471 2024-09-11T01:15:16.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:15:16.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 40 20 64271 64251 6.27394 4.87109 0.0151726 0.0124471 2024-09-11T01:15:16.741 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:15:16.739+0000 7f058360c640 -1 osd.1 26 log_to_monitors true 2024-09-11T01:15:17.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:no valid command found; 10 closest matches: 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:0 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:1 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:2 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:abort 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:assert 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs debug_inject_read_zeros 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs files list 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:bluefs stats 2024-09-11T01:15:17.214 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator dump block 2024-09-11T01:15:17.215 INFO:teuthology.orchestra.run.smithi039.stderr:bluestore allocator fragmentation block 2024-09-11T01:15:17.215 INFO:teuthology.orchestra.run.smithi039.stderr:admin_socket: invalid command 2024-09-11T01:15:17.216 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:15:17.217 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-11T01:15:17.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 41 20 65374 65354 6.22599 4.30859 0.0257287 0.0125411 2024-09-11T01:15:17.916 INFO:tasks.ceph.osd.1.smithi039.stderr:2024-09-11T01:15:17.913+0000 7f0576ce8640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:15:18.315 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 42 20 65994 65974 6.13541 2.42188 0.0052751 0.0126242 2024-09-11T01:15:19.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 43 20 65994 65974 5.99272 0 - 0.0126242 2024-09-11T01:15:20.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 44 20 65994 65974 5.85652 0 - 0.0126242 2024-09-11T01:15:21.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 45 20 66502 66482 5.77047 0.661458 0.028897 0.0135309 2024-09-11T01:15:22.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T01:15:22.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 46 20 67170 67150 5.70175 2.60938 0.0301091 0.013693 2024-09-11T01:15:22.411 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "ops": [ 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67196 2.7 e39/38 2:e428e078:::benchmark_data_smithi039_41775_object67195:head v 39'4248, mlcod=23'1)", 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.365003+0000", 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.04462381, 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.044647921, 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.412 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365003+0000", 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365003+0000", 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365004+0000", 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0729999999999999e-06 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.413 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365016+0000", 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2126999999999999e-05 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365027+0000", 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0572e-05 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365033+0000", 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.4810000000000003e-06 2024-09-11T01:15:22.414 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365113+0000", 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.9246000000000007e-05 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.365135+0000", 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.2959000000000002e-05 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.415 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67201 2.7 e39/38 2:e3efd8eb:::benchmark_data_smithi039_41775_object67200:head v 39'4249, mlcod=23'1)", 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.372995+0000", 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.036631892999999999, 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.036719335999999998, 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.372995+0000", 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.416 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.372995+0000", 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.372996+0000", 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.1999999999999999e-07 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.373007+0000", 2024-09-11T01:15:22.417 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0900999999999999e-05 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.373020+0000", 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.3259e-05 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.373027+0000", 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.6650000000000003e-06 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.418 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.381331+0000", 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0083030469999999992 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.381373+0000", 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.2450999999999997e-05 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67207 2.7 e39/38 2:e2d69c9d:::benchmark_data_smithi039_41775_object67206:head v 39'4250, mlcod=23'1)", 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.377725+0000", 2024-09-11T01:15:22.419 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.031901442000000002, 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.032042858, 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.377725+0000", 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.377725+0000", 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.420 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.377726+0000", 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.7899999999999997e-07 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.377735+0000", 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.0799999999999995e-06 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.377745+0000", 2024-09-11T01:15:22.421 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 9.5489999999999995e-06 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.377751+0000", 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 6.2469999999999997e-06 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.396263+0000", 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018512047 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.396291+0000", 2024-09-11T01:15:22.422 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.7552e-05 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67213 2.7 e39/38 2:e2d8964d:::benchmark_data_smithi039_41775_object67212:head v 39'4251, mlcod=23'1)", 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.389508+0000", 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.020118496999999999, 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020310477, 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.423 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389508+0000", 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389508+0000", 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389510+0000", 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6220000000000001e-06 2024-09-11T01:15:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389523+0000", 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2986000000000001e-05 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389538+0000", 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5034999999999999e-05 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389547+0000", 2024-09-11T01:15:22.425 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.8869999999999998e-06 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.404717+0000", 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.01517012 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.404773+0000", 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.6298000000000002e-05 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.426 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67214 2.a e39/38 2:590eb5e9:::benchmark_data_smithi039_41775_object67213:head v 39'4195, mlcod=23'3)", 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.389569+0000", 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.020057420999999999, 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.020296901999999999, 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389569+0000", 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.427 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389569+0000", 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389570+0000", 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.5400000000000002e-07 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.428 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389581+0000", 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1148999999999999e-05 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389589+0000", 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.8599999999999993e-06 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389593+0000", 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.578e-06 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.429 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389645+0000", 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.1944000000000003e-05 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.389669+0000", 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.4255000000000001e-05 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.430 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67216 2.f e39/38 2:fcf65d74:::benchmark_data_smithi039_41775_object67215:head v 39'4189, mlcod=0'0)", 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.391299+0000", 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.018327705, 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.018616054, 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.391299+0000", 2024-09-11T01:15:22.431 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.391299+0000", 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.391300+0000", 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.9800000000000002e-07 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.432 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.391311+0000", 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.098e-05 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.391323+0000", 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2264e-05 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.391331+0000", 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 7.977e-06 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.433 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.402948+0000", 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.011617021 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.402972+0000", 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.3988000000000001e-05 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67218 2.9 e39/38 2:996f7774:::benchmark_data_smithi039_41775_object67217:head v 39'4360, mlcod=23'4)", 2024-09-11T01:15:22.434 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.403056+0000", 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0065709310000000003, 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00691049, 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403056+0000", 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.435 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403056+0000", 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403057+0000", 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.2330000000000001e-06 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.436 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403068+0000", 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.0808999999999999e-05 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403079+0000", 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.1387e-05 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403088+0000", 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.3149999999999992e-06 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.437 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403208+0000", 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.00012023600000000001 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403229+0000", 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 2.1569999999999998e-05 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.438 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67222 2.5 e39/38 2:a0335cf0:::benchmark_data_smithi039_41775_object67221:head v 39'4217, mlcod=23'2)", 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.403114+0000", 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.0065129879999999999, 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0069031020000000004, 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403114+0000", 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.439 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403114+0000", 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403114+0000", 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.3599999999999999e-07 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403123+0000", 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.5620000000000004e-06 2024-09-11T01:15:22.440 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403128+0000", 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 5.7259999999999999e-06 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.403133+0000", 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 4.4349999999999999e-06 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406852+0000", 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0037191839999999999 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406885+0000", 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2877999999999999e-05 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67223 2.d e39/38)", 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.406598+0000", 2024-09-11T01:15:22.442 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.00302929, 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.003469073, 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "queued for pg", 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406598+0000", 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406598+0000", 2024-09-11T01:15:22.443 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406599+0000", 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7850000000000001e-06 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406617+0000", 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.7249e-05 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.444 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406635+0000", 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.8343e-05 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.406650+0000", 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.5105e-05 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: "description": "osd_repop(client.5047.0:67188 2.f e39/38 2:fcb56121:::benchmark_data_smithi039_41775_object67187:head v 39'4188, mlcod=0'0)", 2024-09-11T01:15:22.445 INFO:teuthology.orchestra.run.smithi039.stdout: "initiated_at": "2024-09-11T01:15:22.349367+0000", 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "age": 0.060259782999999997, 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.060742328999999998, 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "continuous": false, 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "type_data": { 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "flag_point": "started", 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "events": [ 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "initiated", 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.349367+0000", 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "header_read", 2024-09-11T01:15:22.446 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.349367+0000", 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "throttled", 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.349369+0000", 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.584e-06 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "all_read", 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.349385+0000", 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.6311999999999999e-05 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.447 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "dispatched", 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.349398+0000", 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 1.327e-05 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "queued_for_pg", 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.349407+0000", 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 8.7069999999999998e-06 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "reached_pg", 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.352308+0000", 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 0.0029015199999999999 2024-09-11T01:15:22.448 INFO:teuthology.orchestra.run.smithi039.stdout: }, 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: { 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: "event": "started", 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: "time": "2024-09-11T01:15:22.352341+0000", 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: "duration": 3.2141999999999998e-05 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: ] 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: } 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: ], 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout: "num_ops": 10 2024-09-11T01:15:22.449 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:15:22.450 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.1 to be up 2024-09-11T01:15:22.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:15:23.058 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:23.059 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":39,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:15:19.963013+0000","last_up_change":"2024-09-11T01:15:17.956478+0000","last_in_change":"2024-09-11T01:15:11.886947+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-11T01:13:44.644516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":38,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2566795661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":2566795661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":2566795661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":2566795661}]},"public_addr":"172.21.15.39:6800/2566795661","cluster_addr":"172.21.15.39:6801/2566795661","heartbeat_back_addr":"172.21.15.39:6803/2566795661","heartbeat_front_addr":"172.21.15.39:6802/2566795661","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":37,"up_thru":38,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1018236417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":1018236417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":1018236417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1018236417}]},"public_addr":"172.21.15.39:6808/1018236417","cluster_addr":"172.21.15.39:6809/1018236417","heartbeat_back_addr":"172.21.15.39:6811/1018236417","heartbeat_front_addr":"172.21.15.39:6810/1018236417","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":38,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":833967465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":833967465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":833967465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":833967465}]},"public_addr":"172.21.15.39:6804/833967465","cluster_addr":"172.21.15.39:6805/833967465","heartbeat_back_addr":"172.21.15.39:6807/833967465","heartbeat_front_addr":"172.21.15.39:6806/833967465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:15:23.089 INFO:tasks.lost_unfound.ceph_manager:osd.1 is up 2024-09-11T01:15:23.089 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-11T01:15:23.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:23.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 47 20 67748 67728 5.62847 2.25781 0.0145572 0.0138693 2024-09-11T01:15:23.664 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:23.665 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:23.689 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-11T01:15:22.418997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270275313,"num_objects":65996,"num_object_clones":0,"num_object_copies":131992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65996,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25131,"num_write_kb":100518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66069,"ondisk_log_size":66069,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":896036,"kb_used_data":531688,"kb_used_omap":12,"kb_used_meta":364275,"kb_avail":282219484,"statfs":{"total":289910292480,"available":288992751616,"internally_reserved":0,"allocated":544448512,"data_stored":543628940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":373017607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,0,0,0,0,15,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-46896,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.457083"},"pg_stats":[{"pgid":"2.f","version":"36'4123","reported_seq":4180,"reported_epoch":37,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:17.967735+0000","last_change":"2024-09-11T01:15:17.967735+0000","last_active":"2024-09-11T01:15:17.967735+0000","last_peered":"2024-09-11T01:15:17.959089+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:13.939161+0000","last_became_peered":"2024-09-11T01:15:13.939161+0000","last_unstale":"2024-09-11T01:15:17.967735+0000","last_undegraded":"2024-09-11T01:15:17.967735+0000","last_fullsized":"2024-09-11T01:15:17.967735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16887808,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'4094","reported_seq":4162,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.976977+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:18.976977+0000","last_peered":"2024-09-11T01:15:18.976977+0000","last_clean":"2024-09-11T01:15:18.976977+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:18.976977+0000","last_undegraded":"2024-09-11T01:15:18.976977+0000","last_fullsized":"2024-09-11T01:15:18.976977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745414,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'4009","reported_seq":4076,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973540+0000","last_change":"2024-09-11T01:15:17.969362+0000","last_active":"2024-09-11T01:15:17.969362+0000","last_peered":"2024-09-11T01:15:17.960620+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:13.948014+0000","last_became_peered":"2024-09-11T01:15:13.948014+0000","last_unstale":"2024-09-11T01:15:18.973540+0000","last_undegraded":"2024-09-11T01:15:18.973540+0000","last_fullsized":"2024-09-11T01:15:18.973540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4009,"log_dups_size":0,"ondisk_log_size":4009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404480,"num_objects":4005,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'4070","reported_seq":4103,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:18.975892+0000","last_peered":"2024-09-11T01:15:18.975892+0000","last_clean":"2024-09-11T01:15:18.975892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:18.975892+0000","last_undegraded":"2024-09-11T01:15:18.975892+0000","last_fullsized":"2024-09-11T01:15:18.975892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16670720,"num_objects":4070,"num_object_clones":0,"num_object_copies":8140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'4102","reported_seq":4211,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.971368+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:18.971368+0000","last_peered":"2024-09-11T01:15:18.971368+0000","last_clean":"2024-09-11T01:15:18.971368+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:18.971368+0000","last_undegraded":"2024-09-11T01:15:18.971368+0000","last_fullsized":"2024-09-11T01:15:18.971368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4102,"log_dups_size":0,"ondisk_log_size":4102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":900,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'4119","reported_seq":4237,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971274+0000","last_change":"2024-09-11T01:15:17.965111+0000","last_active":"2024-09-11T01:15:17.965111+0000","last_peered":"2024-09-11T01:15:17.953735+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:13.934837+0000","last_became_peered":"2024-09-11T01:15:13.934837+0000","last_unstale":"2024-09-11T01:15:18.971274+0000","last_undegraded":"2024-09-11T01:15:18.971274+0000","last_fullsized":"2024-09-11T01:15:18.971274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16851427,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":607,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'4287","reported_seq":4404,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969996+0000","last_change":"2024-09-11T01:15:17.965901+0000","last_active":"2024-09-11T01:15:17.965901+0000","last_peered":"2024-09-11T01:15:17.937255+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:13.950232+0000","last_became_peered":"2024-09-11T01:15:13.950232+0000","last_unstale":"2024-09-11T01:15:18.969996+0000","last_undegraded":"2024-09-11T01:15:18.969996+0000","last_fullsized":"2024-09-11T01:15:18.969996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535942,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1040,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'4122","reported_seq":4229,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.970354+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:18.970354+0000","last_peered":"2024-09-11T01:15:18.970354+0000","last_clean":"2024-09-11T01:15:18.970354+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:18.970354+0000","last_undegraded":"2024-09-11T01:15:18.970354+0000","last_fullsized":"2024-09-11T01:15:18.970354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4122,"log_dups_size":0,"ondisk_log_size":4122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":4300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"36'4130","reported_seq":4196,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973842+0000","last_change":"2024-09-11T01:15:17.969236+0000","last_active":"2024-09-11T01:15:17.969236+0000","last_peered":"2024-09-11T01:15:17.939231+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:13.930440+0000","last_became_peered":"2024-09-11T01:15:13.930440+0000","last_unstale":"2024-09-11T01:15:18.973842+0000","last_undegraded":"2024-09-11T01:15:18.973842+0000","last_fullsized":"2024-09-11T01:15:18.973842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16888291,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1997,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975023+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:18.975023+0000","last_peered":"2024-09-11T01:15:18.975023+0000","last_clean":"2024-09-11T01:15:18.975023+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:18.975023+0000","last_undegraded":"2024-09-11T01:15:18.975023+0000","last_fullsized":"2024-09-11T01:15:18.975023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'4113","reported_seq":4226,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971574+0000","last_change":"2024-09-11T01:15:17.970681+0000","last_active":"2024-09-11T01:15:17.970681+0000","last_peered":"2024-09-11T01:15:17.947972+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:13.934255+0000","last_became_peered":"2024-09-11T01:15:13.934255+0000","last_unstale":"2024-09-11T01:15:18.971574+0000","last_undegraded":"2024-09-11T01:15:18.971574+0000","last_fullsized":"2024-09-11T01:15:18.971574+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16814080,"num_objects":4105,"num_object_clones":0,"num_object_copies":8210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":477,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":78,"reported_epoch":37,"state":"peering","last_fresh":"2024-09-11T01:15:17.964737+0000","last_change":"2024-09-11T01:15:18.970513+0000","last_active":"2024-09-11T01:15:17.964737+0000","last_peered":"2024-09-11T01:15:16.966390+0000","last_clean":"2024-09-11T01:15:16.966390+0000","last_became_active":"2024-09-11T01:15:13.942375+0000","last_became_peered":"2024-09-11T01:15:13.942375+0000","last_unstale":"2024-09-11T01:15:17.964737+0000","last_undegraded":"2024-09-11T01:15:17.964737+0000","last_fullsized":"2024-09-11T01:15:17.964737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'4031","reported_seq":4109,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.974728+0000","last_change":"2024-09-11T01:15:17.969719+0000","last_active":"2024-09-11T01:15:17.969719+0000","last_peered":"2024-09-11T01:15:17.951087+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:13.934551+0000","last_became_peered":"2024-09-11T01:15:13.934551+0000","last_unstale":"2024-09-11T01:15:18.974728+0000","last_undegraded":"2024-09-11T01:15:18.974728+0000","last_fullsized":"2024-09-11T01:15:18.974728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4031,"log_dups_size":0,"ondisk_log_size":4031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490979,"num_objects":4027,"num_object_clones":0,"num_object_copies":8054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1800,"num_write_kb":7200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"36'4183","reported_seq":4260,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.972787+0000","last_change":"2024-09-11T01:15:17.968291+0000","last_active":"2024-09-11T01:15:17.968291+0000","last_peered":"2024-09-11T01:15:17.954818+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:13.933825+0000","last_became_peered":"2024-09-11T01:15:13.933825+0000","last_unstale":"2024-09-11T01:15:18.972787+0000","last_undegraded":"2024-09-11T01:15:18.972787+0000","last_fullsized":"2024-09-11T01:15:18.972787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1820,"num_write_kb":7280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'4166","reported_seq":4280,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969509+0000","last_change":"2024-09-11T01:15:17.965303+0000","last_active":"2024-09-11T01:15:17.965303+0000","last_peered":"2024-09-11T01:15:17.959258+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:13.934967+0000","last_became_peered":"2024-09-11T01:15:13.934967+0000","last_unstale":"2024-09-11T01:15:18.969509+0000","last_undegraded":"2024-09-11T01:15:18.969509+0000","last_fullsized":"2024-09-11T01:15:18.969509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17055744,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"36'4200","reported_seq":4318,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969953+0000","last_change":"2024-09-11T01:15:17.965409+0000","last_active":"2024-09-11T01:15:17.965409+0000","last_peered":"2024-09-11T01:15:17.926987+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:13.935960+0000","last_became_peered":"2024-09-11T01:15:13.935960+0000","last_unstale":"2024-09-11T01:15:18.969953+0000","last_undegraded":"2024-09-11T01:15:18.969953+0000","last_fullsized":"2024-09-11T01:15:18.969953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4200,"log_dups_size":0,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":2372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.973790+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:18.973790+0000","last_peered":"2024-09-11T01:15:18.973790+0000","last_clean":"2024-09-11T01:15:18.973790+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:18.973790+0000","last_undegraded":"2024-09-11T01:15:18.973790+0000","last_fullsized":"2024-09-11T01:15:18.973790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'4152","reported_seq":4269,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971059+0000","last_change":"2024-09-11T01:15:17.969308+0000","last_active":"2024-09-11T01:15:17.969308+0000","last_peered":"2024-09-11T01:15:17.959501+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:13.949740+0000","last_became_peered":"2024-09-11T01:15:13.949740+0000","last_unstale":"2024-09-11T01:15:18.971059+0000","last_undegraded":"2024-09-11T01:15:18.971059+0000","last_fullsized":"2024-09-11T01:15:18.971059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4152,"log_dups_size":0,"ondisk_log_size":4152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16990208,"num_objects":4148,"num_object_clones":0,"num_object_copies":8296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"36'4165","reported_seq":4284,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.970846+0000","last_change":"2024-09-11T01:15:17.969592+0000","last_active":"2024-09-11T01:15:17.969592+0000","last_peered":"2024-09-11T01:15:17.933954+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:13.934865+0000","last_became_peered":"2024-09-11T01:15:13.934865+0000","last_unstale":"2024-09-11T01:15:18.970846+0000","last_undegraded":"2024-09-11T01:15:18.970846+0000","last_fullsized":"2024-09-11T01:15:18.970846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4165,"log_dups_size":0,"ondisk_log_size":4165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17011654,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":822,"num_write_kb":3288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270275294,"num_objects":65994,"num_object_clones":0,"num_object_copies":131988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65994,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25129,"num_write_kb":100516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543809536,"data_stored":543654177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66066,"ondisk_log_size":66066,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412596,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93959244,"statfs":{"total":96636764160,"available":96214265856,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789954,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70908,"kb_used_data":3112,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94300932,"statfs":{"total":96636764160,"available":96564154368,"internally_reserved":0,"allocated":3186688,"data_stored":2878738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":69398191},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412532,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959308,"statfs":{"total":96636764160,"available":96214331392,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":3111781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:23.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:24.250 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:24.250 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:24.277 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-11T01:15:22.418997+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270275313,"num_objects":65996,"num_object_clones":0,"num_object_copies":131992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65996,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25131,"num_write_kb":100518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66069,"ondisk_log_size":66069,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":896036,"kb_used_data":531688,"kb_used_omap":12,"kb_used_meta":364275,"kb_avail":282219484,"statfs":{"total":289910292480,"available":288992751616,"internally_reserved":0,"allocated":544448512,"data_stored":543628940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":373017607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,0,0,0,0,15,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-46896,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.457083"},"pg_stats":[{"pgid":"2.f","version":"36'4123","reported_seq":4180,"reported_epoch":37,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:17.967735+0000","last_change":"2024-09-11T01:15:17.967735+0000","last_active":"2024-09-11T01:15:17.967735+0000","last_peered":"2024-09-11T01:15:17.959089+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:13.939161+0000","last_became_peered":"2024-09-11T01:15:13.939161+0000","last_unstale":"2024-09-11T01:15:17.967735+0000","last_undegraded":"2024-09-11T01:15:17.967735+0000","last_fullsized":"2024-09-11T01:15:17.967735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16887808,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'4094","reported_seq":4162,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.976977+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:18.976977+0000","last_peered":"2024-09-11T01:15:18.976977+0000","last_clean":"2024-09-11T01:15:18.976977+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:18.976977+0000","last_undegraded":"2024-09-11T01:15:18.976977+0000","last_fullsized":"2024-09-11T01:15:18.976977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745414,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'4009","reported_seq":4076,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973540+0000","last_change":"2024-09-11T01:15:17.969362+0000","last_active":"2024-09-11T01:15:17.969362+0000","last_peered":"2024-09-11T01:15:17.960620+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:13.948014+0000","last_became_peered":"2024-09-11T01:15:13.948014+0000","last_unstale":"2024-09-11T01:15:18.973540+0000","last_undegraded":"2024-09-11T01:15:18.973540+0000","last_fullsized":"2024-09-11T01:15:18.973540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4009,"log_dups_size":0,"ondisk_log_size":4009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404480,"num_objects":4005,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'4070","reported_seq":4103,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:18.975892+0000","last_peered":"2024-09-11T01:15:18.975892+0000","last_clean":"2024-09-11T01:15:18.975892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:18.975892+0000","last_undegraded":"2024-09-11T01:15:18.975892+0000","last_fullsized":"2024-09-11T01:15:18.975892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16670720,"num_objects":4070,"num_object_clones":0,"num_object_copies":8140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'4102","reported_seq":4211,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.971368+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:18.971368+0000","last_peered":"2024-09-11T01:15:18.971368+0000","last_clean":"2024-09-11T01:15:18.971368+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:18.971368+0000","last_undegraded":"2024-09-11T01:15:18.971368+0000","last_fullsized":"2024-09-11T01:15:18.971368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4102,"log_dups_size":0,"ondisk_log_size":4102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":900,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'4119","reported_seq":4237,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971274+0000","last_change":"2024-09-11T01:15:17.965111+0000","last_active":"2024-09-11T01:15:17.965111+0000","last_peered":"2024-09-11T01:15:17.953735+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:13.934837+0000","last_became_peered":"2024-09-11T01:15:13.934837+0000","last_unstale":"2024-09-11T01:15:18.971274+0000","last_undegraded":"2024-09-11T01:15:18.971274+0000","last_fullsized":"2024-09-11T01:15:18.971274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16851427,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":607,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'4287","reported_seq":4404,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969996+0000","last_change":"2024-09-11T01:15:17.965901+0000","last_active":"2024-09-11T01:15:17.965901+0000","last_peered":"2024-09-11T01:15:17.937255+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:13.950232+0000","last_became_peered":"2024-09-11T01:15:13.950232+0000","last_unstale":"2024-09-11T01:15:18.969996+0000","last_undegraded":"2024-09-11T01:15:18.969996+0000","last_fullsized":"2024-09-11T01:15:18.969996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535942,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1040,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'4122","reported_seq":4229,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.970354+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:18.970354+0000","last_peered":"2024-09-11T01:15:18.970354+0000","last_clean":"2024-09-11T01:15:18.970354+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:18.970354+0000","last_undegraded":"2024-09-11T01:15:18.970354+0000","last_fullsized":"2024-09-11T01:15:18.970354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4122,"log_dups_size":0,"ondisk_log_size":4122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":4300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"36'4130","reported_seq":4196,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973842+0000","last_change":"2024-09-11T01:15:17.969236+0000","last_active":"2024-09-11T01:15:17.969236+0000","last_peered":"2024-09-11T01:15:17.939231+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:13.930440+0000","last_became_peered":"2024-09-11T01:15:13.930440+0000","last_unstale":"2024-09-11T01:15:18.973842+0000","last_undegraded":"2024-09-11T01:15:18.973842+0000","last_fullsized":"2024-09-11T01:15:18.973842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16888291,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1997,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975023+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:18.975023+0000","last_peered":"2024-09-11T01:15:18.975023+0000","last_clean":"2024-09-11T01:15:18.975023+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:18.975023+0000","last_undegraded":"2024-09-11T01:15:18.975023+0000","last_fullsized":"2024-09-11T01:15:18.975023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'4113","reported_seq":4226,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971574+0000","last_change":"2024-09-11T01:15:17.970681+0000","last_active":"2024-09-11T01:15:17.970681+0000","last_peered":"2024-09-11T01:15:17.947972+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:13.934255+0000","last_became_peered":"2024-09-11T01:15:13.934255+0000","last_unstale":"2024-09-11T01:15:18.971574+0000","last_undegraded":"2024-09-11T01:15:18.971574+0000","last_fullsized":"2024-09-11T01:15:18.971574+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16814080,"num_objects":4105,"num_object_clones":0,"num_object_copies":8210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":477,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":78,"reported_epoch":37,"state":"peering","last_fresh":"2024-09-11T01:15:17.964737+0000","last_change":"2024-09-11T01:15:18.970513+0000","last_active":"2024-09-11T01:15:17.964737+0000","last_peered":"2024-09-11T01:15:16.966390+0000","last_clean":"2024-09-11T01:15:16.966390+0000","last_became_active":"2024-09-11T01:15:13.942375+0000","last_became_peered":"2024-09-11T01:15:13.942375+0000","last_unstale":"2024-09-11T01:15:17.964737+0000","last_undegraded":"2024-09-11T01:15:17.964737+0000","last_fullsized":"2024-09-11T01:15:17.964737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'4031","reported_seq":4109,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.974728+0000","last_change":"2024-09-11T01:15:17.969719+0000","last_active":"2024-09-11T01:15:17.969719+0000","last_peered":"2024-09-11T01:15:17.951087+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:13.934551+0000","last_became_peered":"2024-09-11T01:15:13.934551+0000","last_unstale":"2024-09-11T01:15:18.974728+0000","last_undegraded":"2024-09-11T01:15:18.974728+0000","last_fullsized":"2024-09-11T01:15:18.974728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4031,"log_dups_size":0,"ondisk_log_size":4031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490979,"num_objects":4027,"num_object_clones":0,"num_object_copies":8054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1800,"num_write_kb":7200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"36'4183","reported_seq":4260,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.972787+0000","last_change":"2024-09-11T01:15:17.968291+0000","last_active":"2024-09-11T01:15:17.968291+0000","last_peered":"2024-09-11T01:15:17.954818+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:13.933825+0000","last_became_peered":"2024-09-11T01:15:13.933825+0000","last_unstale":"2024-09-11T01:15:18.972787+0000","last_undegraded":"2024-09-11T01:15:18.972787+0000","last_fullsized":"2024-09-11T01:15:18.972787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1820,"num_write_kb":7280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'4166","reported_seq":4280,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969509+0000","last_change":"2024-09-11T01:15:17.965303+0000","last_active":"2024-09-11T01:15:17.965303+0000","last_peered":"2024-09-11T01:15:17.959258+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:13.934967+0000","last_became_peered":"2024-09-11T01:15:13.934967+0000","last_unstale":"2024-09-11T01:15:18.969509+0000","last_undegraded":"2024-09-11T01:15:18.969509+0000","last_fullsized":"2024-09-11T01:15:18.969509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17055744,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"36'4200","reported_seq":4318,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969953+0000","last_change":"2024-09-11T01:15:17.965409+0000","last_active":"2024-09-11T01:15:17.965409+0000","last_peered":"2024-09-11T01:15:17.926987+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:13.935960+0000","last_became_peered":"2024-09-11T01:15:13.935960+0000","last_unstale":"2024-09-11T01:15:18.969953+0000","last_undegraded":"2024-09-11T01:15:18.969953+0000","last_fullsized":"2024-09-11T01:15:18.969953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4200,"log_dups_size":0,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":2372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.973790+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:18.973790+0000","last_peered":"2024-09-11T01:15:18.973790+0000","last_clean":"2024-09-11T01:15:18.973790+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:18.973790+0000","last_undegraded":"2024-09-11T01:15:18.973790+0000","last_fullsized":"2024-09-11T01:15:18.973790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'4152","reported_seq":4269,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971059+0000","last_change":"2024-09-11T01:15:17.969308+0000","last_active":"2024-09-11T01:15:17.969308+0000","last_peered":"2024-09-11T01:15:17.959501+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:13.949740+0000","last_became_peered":"2024-09-11T01:15:13.949740+0000","last_unstale":"2024-09-11T01:15:18.971059+0000","last_undegraded":"2024-09-11T01:15:18.971059+0000","last_fullsized":"2024-09-11T01:15:18.971059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4152,"log_dups_size":0,"ondisk_log_size":4152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16990208,"num_objects":4148,"num_object_clones":0,"num_object_copies":8296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"36'4165","reported_seq":4284,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.970846+0000","last_change":"2024-09-11T01:15:17.969592+0000","last_active":"2024-09-11T01:15:17.969592+0000","last_peered":"2024-09-11T01:15:17.933954+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:13.934865+0000","last_became_peered":"2024-09-11T01:15:13.934865+0000","last_unstale":"2024-09-11T01:15:18.970846+0000","last_undegraded":"2024-09-11T01:15:18.970846+0000","last_fullsized":"2024-09-11T01:15:18.970846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4165,"log_dups_size":0,"ondisk_log_size":4165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17011654,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":822,"num_write_kb":3288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270275294,"num_objects":65994,"num_object_clones":0,"num_object_copies":131988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65994,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25129,"num_write_kb":100516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543809536,"data_stored":543654177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66066,"ondisk_log_size":66066,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412596,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93959244,"statfs":{"total":96636764160,"available":96214265856,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789954,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70908,"kb_used_data":3112,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94300932,"statfs":{"total":96636764160,"available":96564154368,"internally_reserved":0,"allocated":3186688,"data_stored":2878738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":69398191},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412532,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959308,"statfs":{"total":96636764160,"available":96214331392,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":3111781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:24.278 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:24.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 48 20 68325 68305 5.55817 2.25391 0.0666882 0.0140472 2024-09-11T01:15:24.876 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:24.879 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:24.905 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-11T01:15:24.421993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270275313,"num_objects":65996,"num_object_clones":0,"num_object_copies":131992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65996,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25131,"num_write_kb":100518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66069,"ondisk_log_size":66069,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":896036,"kb_used_data":531688,"kb_used_omap":12,"kb_used_meta":364275,"kb_avail":282219484,"statfs":{"total":289910292480,"available":288992751616,"internally_reserved":0,"allocated":544448512,"data_stored":543628940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":373017607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,0,0,0,0,15,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-46896,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.456529"},"pg_stats":[{"pgid":"2.f","version":"36'4123","reported_seq":4180,"reported_epoch":37,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:17.967735+0000","last_change":"2024-09-11T01:15:17.967735+0000","last_active":"2024-09-11T01:15:17.967735+0000","last_peered":"2024-09-11T01:15:17.959089+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:13.939161+0000","last_became_peered":"2024-09-11T01:15:13.939161+0000","last_unstale":"2024-09-11T01:15:17.967735+0000","last_undegraded":"2024-09-11T01:15:17.967735+0000","last_fullsized":"2024-09-11T01:15:17.967735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16887808,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'4094","reported_seq":4162,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.976977+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:18.976977+0000","last_peered":"2024-09-11T01:15:18.976977+0000","last_clean":"2024-09-11T01:15:18.976977+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:18.976977+0000","last_undegraded":"2024-09-11T01:15:18.976977+0000","last_fullsized":"2024-09-11T01:15:18.976977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745414,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'4009","reported_seq":4076,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973540+0000","last_change":"2024-09-11T01:15:17.969362+0000","last_active":"2024-09-11T01:15:17.969362+0000","last_peered":"2024-09-11T01:15:17.960620+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:13.948014+0000","last_became_peered":"2024-09-11T01:15:13.948014+0000","last_unstale":"2024-09-11T01:15:18.973540+0000","last_undegraded":"2024-09-11T01:15:18.973540+0000","last_fullsized":"2024-09-11T01:15:18.973540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4009,"log_dups_size":0,"ondisk_log_size":4009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404480,"num_objects":4005,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'4070","reported_seq":4103,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:18.975892+0000","last_peered":"2024-09-11T01:15:18.975892+0000","last_clean":"2024-09-11T01:15:18.975892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:18.975892+0000","last_undegraded":"2024-09-11T01:15:18.975892+0000","last_fullsized":"2024-09-11T01:15:18.975892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16670720,"num_objects":4070,"num_object_clones":0,"num_object_copies":8140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'4102","reported_seq":4211,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.971368+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:18.971368+0000","last_peered":"2024-09-11T01:15:18.971368+0000","last_clean":"2024-09-11T01:15:18.971368+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:18.971368+0000","last_undegraded":"2024-09-11T01:15:18.971368+0000","last_fullsized":"2024-09-11T01:15:18.971368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4102,"log_dups_size":0,"ondisk_log_size":4102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":900,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'4119","reported_seq":4237,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971274+0000","last_change":"2024-09-11T01:15:17.965111+0000","last_active":"2024-09-11T01:15:17.965111+0000","last_peered":"2024-09-11T01:15:17.953735+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:13.934837+0000","last_became_peered":"2024-09-11T01:15:13.934837+0000","last_unstale":"2024-09-11T01:15:18.971274+0000","last_undegraded":"2024-09-11T01:15:18.971274+0000","last_fullsized":"2024-09-11T01:15:18.971274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16851427,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":607,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'4287","reported_seq":4404,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969996+0000","last_change":"2024-09-11T01:15:17.965901+0000","last_active":"2024-09-11T01:15:17.965901+0000","last_peered":"2024-09-11T01:15:17.937255+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:13.950232+0000","last_became_peered":"2024-09-11T01:15:13.950232+0000","last_unstale":"2024-09-11T01:15:18.969996+0000","last_undegraded":"2024-09-11T01:15:18.969996+0000","last_fullsized":"2024-09-11T01:15:18.969996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535942,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1040,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'4122","reported_seq":4229,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.970354+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:18.970354+0000","last_peered":"2024-09-11T01:15:18.970354+0000","last_clean":"2024-09-11T01:15:18.970354+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:18.970354+0000","last_undegraded":"2024-09-11T01:15:18.970354+0000","last_fullsized":"2024-09-11T01:15:18.970354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4122,"log_dups_size":0,"ondisk_log_size":4122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":4300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"36'4130","reported_seq":4196,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973842+0000","last_change":"2024-09-11T01:15:17.969236+0000","last_active":"2024-09-11T01:15:17.969236+0000","last_peered":"2024-09-11T01:15:17.939231+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:13.930440+0000","last_became_peered":"2024-09-11T01:15:13.930440+0000","last_unstale":"2024-09-11T01:15:18.973842+0000","last_undegraded":"2024-09-11T01:15:18.973842+0000","last_fullsized":"2024-09-11T01:15:18.973842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16888291,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1997,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975023+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:18.975023+0000","last_peered":"2024-09-11T01:15:18.975023+0000","last_clean":"2024-09-11T01:15:18.975023+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:18.975023+0000","last_undegraded":"2024-09-11T01:15:18.975023+0000","last_fullsized":"2024-09-11T01:15:18.975023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'4113","reported_seq":4226,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971574+0000","last_change":"2024-09-11T01:15:17.970681+0000","last_active":"2024-09-11T01:15:17.970681+0000","last_peered":"2024-09-11T01:15:17.947972+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:13.934255+0000","last_became_peered":"2024-09-11T01:15:13.934255+0000","last_unstale":"2024-09-11T01:15:18.971574+0000","last_undegraded":"2024-09-11T01:15:18.971574+0000","last_fullsized":"2024-09-11T01:15:18.971574+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16814080,"num_objects":4105,"num_object_clones":0,"num_object_copies":8210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":477,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":78,"reported_epoch":37,"state":"peering","last_fresh":"2024-09-11T01:15:17.964737+0000","last_change":"2024-09-11T01:15:18.970513+0000","last_active":"2024-09-11T01:15:17.964737+0000","last_peered":"2024-09-11T01:15:16.966390+0000","last_clean":"2024-09-11T01:15:16.966390+0000","last_became_active":"2024-09-11T01:15:13.942375+0000","last_became_peered":"2024-09-11T01:15:13.942375+0000","last_unstale":"2024-09-11T01:15:17.964737+0000","last_undegraded":"2024-09-11T01:15:17.964737+0000","last_fullsized":"2024-09-11T01:15:17.964737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'4031","reported_seq":4109,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.974728+0000","last_change":"2024-09-11T01:15:17.969719+0000","last_active":"2024-09-11T01:15:17.969719+0000","last_peered":"2024-09-11T01:15:17.951087+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:13.934551+0000","last_became_peered":"2024-09-11T01:15:13.934551+0000","last_unstale":"2024-09-11T01:15:18.974728+0000","last_undegraded":"2024-09-11T01:15:18.974728+0000","last_fullsized":"2024-09-11T01:15:18.974728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4031,"log_dups_size":0,"ondisk_log_size":4031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490979,"num_objects":4027,"num_object_clones":0,"num_object_copies":8054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1800,"num_write_kb":7200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"36'4183","reported_seq":4260,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.972787+0000","last_change":"2024-09-11T01:15:17.968291+0000","last_active":"2024-09-11T01:15:17.968291+0000","last_peered":"2024-09-11T01:15:17.954818+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:13.933825+0000","last_became_peered":"2024-09-11T01:15:13.933825+0000","last_unstale":"2024-09-11T01:15:18.972787+0000","last_undegraded":"2024-09-11T01:15:18.972787+0000","last_fullsized":"2024-09-11T01:15:18.972787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1820,"num_write_kb":7280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'4166","reported_seq":4280,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969509+0000","last_change":"2024-09-11T01:15:17.965303+0000","last_active":"2024-09-11T01:15:17.965303+0000","last_peered":"2024-09-11T01:15:17.959258+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:13.934967+0000","last_became_peered":"2024-09-11T01:15:13.934967+0000","last_unstale":"2024-09-11T01:15:18.969509+0000","last_undegraded":"2024-09-11T01:15:18.969509+0000","last_fullsized":"2024-09-11T01:15:18.969509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17055744,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"36'4200","reported_seq":4318,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969953+0000","last_change":"2024-09-11T01:15:17.965409+0000","last_active":"2024-09-11T01:15:17.965409+0000","last_peered":"2024-09-11T01:15:17.926987+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:13.935960+0000","last_became_peered":"2024-09-11T01:15:13.935960+0000","last_unstale":"2024-09-11T01:15:18.969953+0000","last_undegraded":"2024-09-11T01:15:18.969953+0000","last_fullsized":"2024-09-11T01:15:18.969953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4200,"log_dups_size":0,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":2372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.973790+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:18.973790+0000","last_peered":"2024-09-11T01:15:18.973790+0000","last_clean":"2024-09-11T01:15:18.973790+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:18.973790+0000","last_undegraded":"2024-09-11T01:15:18.973790+0000","last_fullsized":"2024-09-11T01:15:18.973790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'4152","reported_seq":4269,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971059+0000","last_change":"2024-09-11T01:15:17.969308+0000","last_active":"2024-09-11T01:15:17.969308+0000","last_peered":"2024-09-11T01:15:17.959501+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:13.949740+0000","last_became_peered":"2024-09-11T01:15:13.949740+0000","last_unstale":"2024-09-11T01:15:18.971059+0000","last_undegraded":"2024-09-11T01:15:18.971059+0000","last_fullsized":"2024-09-11T01:15:18.971059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4152,"log_dups_size":0,"ondisk_log_size":4152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16990208,"num_objects":4148,"num_object_clones":0,"num_object_copies":8296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"36'4165","reported_seq":4284,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.970846+0000","last_change":"2024-09-11T01:15:17.969592+0000","last_active":"2024-09-11T01:15:17.969592+0000","last_peered":"2024-09-11T01:15:17.933954+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:13.934865+0000","last_became_peered":"2024-09-11T01:15:13.934865+0000","last_unstale":"2024-09-11T01:15:18.970846+0000","last_undegraded":"2024-09-11T01:15:18.970846+0000","last_fullsized":"2024-09-11T01:15:18.970846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4165,"log_dups_size":0,"ondisk_log_size":4165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17011654,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":822,"num_write_kb":3288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270275294,"num_objects":65994,"num_object_clones":0,"num_object_copies":131988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65994,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25129,"num_write_kb":100516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543809536,"data_stored":543654177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66066,"ondisk_log_size":66066,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412596,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93959244,"statfs":{"total":96636764160,"available":96214265856,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789954,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70908,"kb_used_data":3112,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94300932,"statfs":{"total":96636764160,"available":96564154368,"internally_reserved":0,"allocated":3186688,"data_stored":2878738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":69398191},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412532,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959308,"statfs":{"total":96636764160,"available":96214331392,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":3111781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:24.906 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:24.906 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "36'4123", 'reported_seq': 4180, 'reported_epoch': 37, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:17.967735+0000', 'last_change': '2024-09-11T01:15:17.967735+0000', 'last_active': '2024-09-11T01:15:17.967735+0000', 'last_peered': '2024-09-11T01:15:17.959089+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:13.939161+0000', 'last_became_peered': '2024-09-11T01:15:13.939161+0000', 'last_unstale': '2024-09-11T01:15:17.967735+0000', 'last_undegraded': '2024-09-11T01:15:17.967735+0000', 'last_fullsized': '2024-09-11T01:15:17.967735+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 4123, 'log_dups_size': 0, 'ondisk_log_size': 4123, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16887808, 'num_objects': 4123, 'num_object_clones': 0, 'num_object_copies': 8246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4123, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4123, 'num_write_kb': 16492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.906 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:24.906 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "36'4009", 'reported_seq': 4076, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.973540+0000', 'last_change': '2024-09-11T01:15:17.969362+0000', 'last_active': '2024-09-11T01:15:17.969362+0000', 'last_peered': '2024-09-11T01:15:17.960620+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:13.948014+0000', 'last_became_peered': '2024-09-11T01:15:13.948014+0000', 'last_unstale': '2024-09-11T01:15:18.973540+0000', 'last_undegraded': '2024-09-11T01:15:18.973540+0000', 'last_fullsized': '2024-09-11T01:15:18.973540+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 4009, 'log_dups_size': 0, 'ondisk_log_size': 4009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16404480, 'num_objects': 4005, 'num_object_clones': 0, 'num_object_copies': 8010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2070, 'num_write_kb': 8280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:24.906 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:24.906 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "36'4119", 'reported_seq': 4237, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.971274+0000', 'last_change': '2024-09-11T01:15:17.965111+0000', 'last_active': '2024-09-11T01:15:17.965111+0000', 'last_peered': '2024-09-11T01:15:17.953735+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:13.934837+0000', 'last_became_peered': '2024-09-11T01:15:13.934837+0000', 'last_unstale': '2024-09-11T01:15:18.971274+0000', 'last_undegraded': '2024-09-11T01:15:18.971274+0000', 'last_fullsized': '2024-09-11T01:15:18.971274+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 4119, 'log_dups_size': 0, 'ondisk_log_size': 4119, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16851427, 'num_objects': 4115, 'num_object_clones': 0, 'num_object_copies': 8230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4115, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 607, 'num_write_kb': 2428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "36'4287", 'reported_seq': 4404, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.969996+0000', 'last_change': '2024-09-11T01:15:17.965901+0000', 'last_active': '2024-09-11T01:15:17.965901+0000', 'last_peered': '2024-09-11T01:15:17.937255+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:13.950232+0000', 'last_became_peered': '2024-09-11T01:15:13.950232+0000', 'last_unstale': '2024-09-11T01:15:18.969996+0000', 'last_undegraded': '2024-09-11T01:15:18.969996+0000', 'last_fullsized': '2024-09-11T01:15:18.969996+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 4287, 'log_dups_size': 0, 'ondisk_log_size': 4287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17535942, 'num_objects': 4283, 'num_object_clones': 0, 'num_object_copies': 8566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4283, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1040, 'num_write_kb': 4160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "36'4130", 'reported_seq': 4196, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.973842+0000', 'last_change': '2024-09-11T01:15:17.969236+0000', 'last_active': '2024-09-11T01:15:17.969236+0000', 'last_peered': '2024-09-11T01:15:17.939231+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:13.930440+0000', 'last_became_peered': '2024-09-11T01:15:13.930440+0000', 'last_unstale': '2024-09-11T01:15:18.973842+0000', 'last_undegraded': '2024-09-11T01:15:18.973842+0000', 'last_fullsized': '2024-09-11T01:15:18.973842+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 4130, 'log_dups_size': 0, 'ondisk_log_size': 4130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16888291, 'num_objects': 4124, 'num_object_clones': 0, 'num_object_copies': 8248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4124, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1997, 'num_write_kb': 7988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "36'4113", 'reported_seq': 4226, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.971574+0000', 'last_change': '2024-09-11T01:15:17.970681+0000', 'last_active': '2024-09-11T01:15:17.970681+0000', 'last_peered': '2024-09-11T01:15:17.947972+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:13.934255+0000', 'last_became_peered': '2024-09-11T01:15:13.934255+0000', 'last_unstale': '2024-09-11T01:15:18.971574+0000', 'last_undegraded': '2024-09-11T01:15:18.971574+0000', 'last_fullsized': '2024-09-11T01:15:18.971574+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 4113, 'log_dups_size': 0, 'ondisk_log_size': 4113, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16814080, 'num_objects': 4105, 'num_object_clones': 0, 'num_object_copies': 8210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 477, 'num_write_kb': 1908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:PG 1.2 is not active+clean 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '1.2', 'version': "14'2", 'reported_seq': 78, 'reported_epoch': 37, 'state': 'peering', 'last_fresh': '2024-09-11T01:15:17.964737+0000', 'last_change': '2024-09-11T01:15:18.970513+0000', 'last_active': '2024-09-11T01:15:17.964737+0000', 'last_peered': '2024-09-11T01:15:16.966390+0000', 'last_clean': '2024-09-11T01:15:16.966390+0000', 'last_became_active': '2024-09-11T01:15:13.942375+0000', 'last_became_peered': '2024-09-11T01:15:13.942375+0000', 'last_unstale': '2024-09-11T01:15:17.964737+0000', 'last_undegraded': '2024-09-11T01:15:17.964737+0000', 'last_fullsized': '2024-09-11T01:15:17.964737+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:35.453339+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:35.453339+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:35.453339+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.000618975, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "36'4031", 'reported_seq': 4109, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.974728+0000', 'last_change': '2024-09-11T01:15:17.969719+0000', 'last_active': '2024-09-11T01:15:17.969719+0000', 'last_peered': '2024-09-11T01:15:17.951087+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:13.934551+0000', 'last_became_peered': '2024-09-11T01:15:13.934551+0000', 'last_unstale': '2024-09-11T01:15:18.974728+0000', 'last_undegraded': '2024-09-11T01:15:18.974728+0000', 'last_fullsized': '2024-09-11T01:15:18.974728+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 4031, 'log_dups_size': 0, 'ondisk_log_size': 4031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16490979, 'num_objects': 4027, 'num_object_clones': 0, 'num_object_copies': 8054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4027, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1800, 'num_write_kb': 7200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:24.907 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "36'4183", 'reported_seq': 4260, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.972787+0000', 'last_change': '2024-09-11T01:15:17.968291+0000', 'last_active': '2024-09-11T01:15:17.968291+0000', 'last_peered': '2024-09-11T01:15:17.954818+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:13.933825+0000', 'last_became_peered': '2024-09-11T01:15:13.933825+0000', 'last_unstale': '2024-09-11T01:15:18.972787+0000', 'last_undegraded': '2024-09-11T01:15:18.972787+0000', 'last_fullsized': '2024-09-11T01:15:18.972787+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 4183, 'log_dups_size': 0, 'ondisk_log_size': 4183, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17108992, 'num_objects': 4177, 'num_object_clones': 0, 'num_object_copies': 8354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1820, 'num_write_kb': 7280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "36'4166", 'reported_seq': 4280, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.969509+0000', 'last_change': '2024-09-11T01:15:17.965303+0000', 'last_active': '2024-09-11T01:15:17.965303+0000', 'last_peered': '2024-09-11T01:15:17.959258+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:13.934967+0000', 'last_became_peered': '2024-09-11T01:15:13.934967+0000', 'last_unstale': '2024-09-11T01:15:18.969509+0000', 'last_undegraded': '2024-09-11T01:15:18.969509+0000', 'last_fullsized': '2024-09-11T01:15:18.969509+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 4166, 'log_dups_size': 0, 'ondisk_log_size': 4166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17055744, 'num_objects': 4164, 'num_object_clones': 0, 'num_object_copies': 8328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 856, 'num_write_kb': 3424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "36'4200", 'reported_seq': 4318, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.969953+0000', 'last_change': '2024-09-11T01:15:17.965409+0000', 'last_active': '2024-09-11T01:15:17.965409+0000', 'last_peered': '2024-09-11T01:15:17.926987+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:13.935960+0000', 'last_became_peered': '2024-09-11T01:15:13.935960+0000', 'last_unstale': '2024-09-11T01:15:18.969953+0000', 'last_undegraded': '2024-09-11T01:15:18.969953+0000', 'last_fullsized': '2024-09-11T01:15:18.969953+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 4200, 'log_dups_size': 0, 'ondisk_log_size': 4200, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17170432, 'num_objects': 4192, 'num_object_clones': 0, 'num_object_copies': 8384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 593, 'num_write_kb': 2372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "36'4152", 'reported_seq': 4269, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.971059+0000', 'last_change': '2024-09-11T01:15:17.969308+0000', 'last_active': '2024-09-11T01:15:17.969308+0000', 'last_peered': '2024-09-11T01:15:17.959501+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:13.949740+0000', 'last_became_peered': '2024-09-11T01:15:13.949740+0000', 'last_unstale': '2024-09-11T01:15:18.971059+0000', 'last_undegraded': '2024-09-11T01:15:18.971059+0000', 'last_fullsized': '2024-09-11T01:15:18.971059+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 4152, 'log_dups_size': 0, 'ondisk_log_size': 4152, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16990208, 'num_objects': 4148, 'num_object_clones': 0, 'num_object_copies': 8296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4148, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 687, 'num_write_kb': 2748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:24.908 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "36'4165", 'reported_seq': 4284, 'reported_epoch': 38, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:15:18.970846+0000', 'last_change': '2024-09-11T01:15:17.969592+0000', 'last_active': '2024-09-11T01:15:17.969592+0000', 'last_peered': '2024-09-11T01:15:17.933954+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:13.934865+0000', 'last_became_peered': '2024-09-11T01:15:13.934865+0000', 'last_unstale': '2024-09-11T01:15:18.970846+0000', 'last_undegraded': '2024-09-11T01:15:18.970846+0000', 'last_fullsized': '2024-09-11T01:15:18.970846+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 4165, 'log_dups_size': 0, 'ondisk_log_size': 4165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17011654, 'num_objects': 4155, 'num_object_clones': 0, 'num_object_copies': 8310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4155, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 822, 'num_write_kb': 3288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:24.909 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:25.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 49 20 68890 68870 5.48977 2.20703 0.0418642 0.0142207 2024-09-11T01:15:25.650 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:25.650 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":126,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"remapped+peering","count":12},{"state_name":"active+clean","count":11},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65996,"data_bytes":270275313,"bytes_used":917540864,"bytes_avail":288992751616,"bytes_total":289910292480,"inactive_pgs_ratio":0.54166668653488159,"write_bytes_sec":2313009,"read_op_per_sec":0,"write_op_per_sec":564},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-11T01:15:10.416849+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:25.674 INFO:tasks.lost_unfound.ceph_manager:no progress seen, keeping timeout for now 2024-09-11T01:15:25.674 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:26.236 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:26.238 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:26.259 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-11T01:15:24.421993+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270275313,"num_objects":65996,"num_object_clones":0,"num_object_copies":131992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65996,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25131,"num_write_kb":100518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66069,"ondisk_log_size":66069,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":896036,"kb_used_data":531688,"kb_used_omap":12,"kb_used_meta":364275,"kb_avail":282219484,"statfs":{"total":289910292480,"available":288992751616,"internally_reserved":0,"allocated":544448512,"data_stored":543628940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":373017607},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4,0,0,0,0,15,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":14934016,"num_objects":3646,"num_object_clones":0,"num_object_copies":7292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-46896,"num_objects_unfound":0,"num_objects_dirty":3646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.456529"},"pg_stats":[{"pgid":"2.f","version":"36'4123","reported_seq":4180,"reported_epoch":37,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:17.967735+0000","last_change":"2024-09-11T01:15:17.967735+0000","last_active":"2024-09-11T01:15:17.967735+0000","last_peered":"2024-09-11T01:15:17.959089+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:13.939161+0000","last_became_peered":"2024-09-11T01:15:13.939161+0000","last_unstale":"2024-09-11T01:15:17.967735+0000","last_undegraded":"2024-09-11T01:15:17.967735+0000","last_fullsized":"2024-09-11T01:15:17.967735+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4123,"log_dups_size":0,"ondisk_log_size":4123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16887808,"num_objects":4123,"num_object_clones":0,"num_object_copies":8246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4123,"num_write_kb":16492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"37'4094","reported_seq":4162,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.976977+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:18.976977+0000","last_peered":"2024-09-11T01:15:18.976977+0000","last_clean":"2024-09-11T01:15:18.976977+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:18.976977+0000","last_undegraded":"2024-09-11T01:15:18.976977+0000","last_fullsized":"2024-09-11T01:15:18.976977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16745414,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'4009","reported_seq":4076,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973540+0000","last_change":"2024-09-11T01:15:17.969362+0000","last_active":"2024-09-11T01:15:17.969362+0000","last_peered":"2024-09-11T01:15:17.960620+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:13.948014+0000","last_became_peered":"2024-09-11T01:15:13.948014+0000","last_unstale":"2024-09-11T01:15:18.973540+0000","last_undegraded":"2024-09-11T01:15:18.973540+0000","last_fullsized":"2024-09-11T01:15:18.973540+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4009,"log_dups_size":0,"ondisk_log_size":4009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404480,"num_objects":4005,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2070,"num_write_kb":8280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"37'4070","reported_seq":4103,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:18.975892+0000","last_peered":"2024-09-11T01:15:18.975892+0000","last_clean":"2024-09-11T01:15:18.975892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:18.975892+0000","last_undegraded":"2024-09-11T01:15:18.975892+0000","last_fullsized":"2024-09-11T01:15:18.975892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16670720,"num_objects":4070,"num_object_clones":0,"num_object_copies":8140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4070,"num_write_kb":16280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"37'4102","reported_seq":4211,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.971368+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:18.971368+0000","last_peered":"2024-09-11T01:15:18.971368+0000","last_clean":"2024-09-11T01:15:18.971368+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:18.971368+0000","last_undegraded":"2024-09-11T01:15:18.971368+0000","last_fullsized":"2024-09-11T01:15:18.971368+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4102,"log_dups_size":0,"ondisk_log_size":4102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16773603,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":900,"num_write_kb":3600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'4119","reported_seq":4237,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971274+0000","last_change":"2024-09-11T01:15:17.965111+0000","last_active":"2024-09-11T01:15:17.965111+0000","last_peered":"2024-09-11T01:15:17.953735+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:13.934837+0000","last_became_peered":"2024-09-11T01:15:13.934837+0000","last_unstale":"2024-09-11T01:15:18.971274+0000","last_undegraded":"2024-09-11T01:15:18.971274+0000","last_fullsized":"2024-09-11T01:15:18.971274+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16851427,"num_objects":4115,"num_object_clones":0,"num_object_copies":8230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":607,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'4287","reported_seq":4404,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969996+0000","last_change":"2024-09-11T01:15:17.965901+0000","last_active":"2024-09-11T01:15:17.965901+0000","last_peered":"2024-09-11T01:15:17.937255+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:13.950232+0000","last_became_peered":"2024-09-11T01:15:13.950232+0000","last_unstale":"2024-09-11T01:15:18.969996+0000","last_undegraded":"2024-09-11T01:15:18.969996+0000","last_fullsized":"2024-09-11T01:15:18.969996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535942,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1040,"num_write_kb":4160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'4122","reported_seq":4229,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.970354+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:18.970354+0000","last_peered":"2024-09-11T01:15:18.970354+0000","last_clean":"2024-09-11T01:15:18.970354+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:18.970354+0000","last_undegraded":"2024-09-11T01:15:18.970354+0000","last_fullsized":"2024-09-11T01:15:18.970354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4122,"log_dups_size":0,"ondisk_log_size":4122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16875520,"num_objects":4120,"num_object_clones":0,"num_object_copies":8240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1075,"num_write_kb":4300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"36'4130","reported_seq":4196,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.973842+0000","last_change":"2024-09-11T01:15:17.969236+0000","last_active":"2024-09-11T01:15:17.969236+0000","last_peered":"2024-09-11T01:15:17.939231+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:13.930440+0000","last_became_peered":"2024-09-11T01:15:13.930440+0000","last_unstale":"2024-09-11T01:15:18.973842+0000","last_undegraded":"2024-09-11T01:15:18.973842+0000","last_fullsized":"2024-09-11T01:15:18.973842+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4130,"log_dups_size":0,"ondisk_log_size":4130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16888291,"num_objects":4124,"num_object_clones":0,"num_object_copies":8248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1997,"num_write_kb":7988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.975023+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:18.975023+0000","last_peered":"2024-09-11T01:15:18.975023+0000","last_clean":"2024-09-11T01:15:18.975023+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:18.975023+0000","last_undegraded":"2024-09-11T01:15:18.975023+0000","last_fullsized":"2024-09-11T01:15:18.975023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'4113","reported_seq":4226,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971574+0000","last_change":"2024-09-11T01:15:17.970681+0000","last_active":"2024-09-11T01:15:17.970681+0000","last_peered":"2024-09-11T01:15:17.947972+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:13.934255+0000","last_became_peered":"2024-09-11T01:15:13.934255+0000","last_unstale":"2024-09-11T01:15:18.971574+0000","last_undegraded":"2024-09-11T01:15:18.971574+0000","last_fullsized":"2024-09-11T01:15:18.971574+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4113,"log_dups_size":0,"ondisk_log_size":4113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16814080,"num_objects":4105,"num_object_clones":0,"num_object_copies":8210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":477,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":78,"reported_epoch":37,"state":"peering","last_fresh":"2024-09-11T01:15:17.964737+0000","last_change":"2024-09-11T01:15:18.970513+0000","last_active":"2024-09-11T01:15:17.964737+0000","last_peered":"2024-09-11T01:15:16.966390+0000","last_clean":"2024-09-11T01:15:16.966390+0000","last_became_active":"2024-09-11T01:15:13.942375+0000","last_became_peered":"2024-09-11T01:15:13.942375+0000","last_unstale":"2024-09-11T01:15:17.964737+0000","last_undegraded":"2024-09-11T01:15:17.964737+0000","last_fullsized":"2024-09-11T01:15:17.964737+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'4031","reported_seq":4109,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.974728+0000","last_change":"2024-09-11T01:15:17.969719+0000","last_active":"2024-09-11T01:15:17.969719+0000","last_peered":"2024-09-11T01:15:17.951087+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:13.934551+0000","last_became_peered":"2024-09-11T01:15:13.934551+0000","last_unstale":"2024-09-11T01:15:18.974728+0000","last_undegraded":"2024-09-11T01:15:18.974728+0000","last_fullsized":"2024-09-11T01:15:18.974728+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4031,"log_dups_size":0,"ondisk_log_size":4031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490979,"num_objects":4027,"num_object_clones":0,"num_object_copies":8054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1800,"num_write_kb":7200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"36'4183","reported_seq":4260,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.972787+0000","last_change":"2024-09-11T01:15:17.968291+0000","last_active":"2024-09-11T01:15:17.968291+0000","last_peered":"2024-09-11T01:15:17.954818+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:13.933825+0000","last_became_peered":"2024-09-11T01:15:13.933825+0000","last_unstale":"2024-09-11T01:15:18.972787+0000","last_undegraded":"2024-09-11T01:15:18.972787+0000","last_fullsized":"2024-09-11T01:15:18.972787+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4183,"log_dups_size":0,"ondisk_log_size":4183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17108992,"num_objects":4177,"num_object_clones":0,"num_object_copies":8354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1820,"num_write_kb":7280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'4166","reported_seq":4280,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969509+0000","last_change":"2024-09-11T01:15:17.965303+0000","last_active":"2024-09-11T01:15:17.965303+0000","last_peered":"2024-09-11T01:15:17.959258+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:13.934967+0000","last_became_peered":"2024-09-11T01:15:13.934967+0000","last_unstale":"2024-09-11T01:15:18.969509+0000","last_undegraded":"2024-09-11T01:15:18.969509+0000","last_fullsized":"2024-09-11T01:15:18.969509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4166,"log_dups_size":0,"ondisk_log_size":4166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17055744,"num_objects":4164,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":856,"num_write_kb":3424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"36'4200","reported_seq":4318,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.969953+0000","last_change":"2024-09-11T01:15:17.965409+0000","last_active":"2024-09-11T01:15:17.965409+0000","last_peered":"2024-09-11T01:15:17.926987+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:13.935960+0000","last_became_peered":"2024-09-11T01:15:13.935960+0000","last_unstale":"2024-09-11T01:15:18.969953+0000","last_undegraded":"2024-09-11T01:15:18.969953+0000","last_fullsized":"2024-09-11T01:15:18.969953+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4200,"log_dups_size":0,"ondisk_log_size":4200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17170432,"num_objects":4192,"num_object_clones":0,"num_object_copies":8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":593,"num_write_kb":2372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.973790+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:18.973790+0000","last_peered":"2024-09-11T01:15:18.973790+0000","last_clean":"2024-09-11T01:15:18.973790+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:18.973790+0000","last_undegraded":"2024-09-11T01:15:18.973790+0000","last_fullsized":"2024-09-11T01:15:18.973790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'4152","reported_seq":4269,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.971059+0000","last_change":"2024-09-11T01:15:17.969308+0000","last_active":"2024-09-11T01:15:17.969308+0000","last_peered":"2024-09-11T01:15:17.959501+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:13.949740+0000","last_became_peered":"2024-09-11T01:15:13.949740+0000","last_unstale":"2024-09-11T01:15:18.971059+0000","last_undegraded":"2024-09-11T01:15:18.971059+0000","last_fullsized":"2024-09-11T01:15:18.971059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4152,"log_dups_size":0,"ondisk_log_size":4152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16990208,"num_objects":4148,"num_object_clones":0,"num_object_copies":8296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"36'4165","reported_seq":4284,"reported_epoch":38,"state":"remapped+peering","last_fresh":"2024-09-11T01:15:18.970846+0000","last_change":"2024-09-11T01:15:17.969592+0000","last_active":"2024-09-11T01:15:17.969592+0000","last_peered":"2024-09-11T01:15:17.933954+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:13.934865+0000","last_became_peered":"2024-09-11T01:15:13.934865+0000","last_unstale":"2024-09-11T01:15:18.970846+0000","last_undegraded":"2024-09-11T01:15:18.970846+0000","last_fullsized":"2024-09-11T01:15:18.970846+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4165,"log_dups_size":0,"ondisk_log_size":4165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17011654,"num_objects":4155,"num_object_clones":0,"num_object_copies":8310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":822,"num_write_kb":3288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270275294,"num_objects":65994,"num_object_clones":0,"num_object_copies":131988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65994,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":25129,"num_write_kb":100516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543809536,"data_stored":543654177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66066,"ondisk_log_size":66066,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412596,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93959244,"statfs":{"total":96636764160,"available":96214265856,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":151842478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,2,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789954,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":70908,"kb_used_data":3112,"kb_used_omap":4,"kb_used_meta":67771,"kb_avail":94300932,"statfs":{"total":96636764160,"available":96564154368,"internally_reserved":0,"allocated":3186688,"data_stored":2878738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":69398191},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182419,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412532,"kb_used_data":264288,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959308,"statfs":{"total":96636764160,"available":96214331392,"internally_reserved":0,"allocated":270630912,"data_stored":270375101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,13],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3194880,"data_stored":3111781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270271198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:26.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 50 20 69460 69440 5.42451 2.22656 0.0114855 0.0143942 2024-09-11T01:15:27.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 51 20 70101 70081 5.36724 2.50391 0.00739699 0.014547 2024-09-11T01:15:28.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 52 20 70738 70718 5.31187 2.48828 0.0273031 0.0146998 2024-09-11T01:15:29.261 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:29.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 53 20 71362 71342 5.25763 2.4375 0.0217654 0.0148508 2024-09-11T01:15:29.812 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:29.812 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:29.828 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-11T01:15:28.422838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280990449,"num_objects":68612,"num_object_clones":0,"num_object_copies":137224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":4029,"num_objects_unfound":0,"num_objects_dirty":68612,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":27747,"num_write_kb":110982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":393235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68685,"ondisk_log_size":68685,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":900808,"kb_used_data":553112,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282214712,"statfs":{"total":289910292480,"available":288987865088,"internally_reserved":0,"allocated":566386688,"data_stored":565560853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":355978247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,10,11,13,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25649152,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":-42867,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6262,"num_write_kb":25048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":389120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.455685"},"pg_stats":[{"pgid":"2.f","version":"39'4270","reported_seq":4333,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.940101+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:24.940101+0000","last_peered":"2024-09-11T01:15:24.940101+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:24.940101+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4270,"log_dups_size":0,"ondisk_log_size":4270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4270,"num_write_kb":17080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":147},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4270","reported_seq":4340,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.607874+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:24.607874+0000","last_peered":"2024-09-11T01:15:24.607874+0000","last_clean":"2024-09-11T01:15:24.607874+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:24.607874+0000","last_undegraded":"2024-09-11T01:15:24.607874+0000","last_fullsized":"2024-09-11T01:15:24.607874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4270,"log_dups_size":0,"ondisk_log_size":4270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17466310,"num_objects":4266,"num_object_clones":0,"num_object_copies":8532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4266,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4161","reported_seq":4233,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.576637+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:24.576637+0000","last_peered":"2024-09-11T01:15:24.576637+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:24.576637+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4161,"log_dups_size":0,"ondisk_log_size":4161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17027072,"num_objects":4157,"num_object_clones":0,"num_object_copies":8314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2222,"num_write_kb":8888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":152}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4237","reported_seq":4272,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.606607+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:24.606607+0000","last_peered":"2024-09-11T01:15:24.606607+0000","last_clean":"2024-09-11T01:15:24.606607+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:24.606607+0000","last_undegraded":"2024-09-11T01:15:24.606607+0000","last_fullsized":"2024-09-11T01:15:24.606607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4237,"log_dups_size":0,"ondisk_log_size":4237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17354752,"num_objects":4237,"num_object_clones":0,"num_object_copies":8474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4292","reported_seq":4403,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.929050+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:24.929050+0000","last_peered":"2024-09-11T01:15:24.929050+0000","last_clean":"2024-09-11T01:15:24.929050+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:24.929050+0000","last_undegraded":"2024-09-11T01:15:24.929050+0000","last_fullsized":"2024-09-11T01:15:24.929050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4292,"log_dups_size":0,"ondisk_log_size":4292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17551843,"num_objects":4286,"num_object_clones":0,"num_object_copies":8572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1090,"num_write_kb":4360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4286","reported_seq":4410,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.909523+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:24.909523+0000","last_peered":"2024-09-11T01:15:24.909523+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:24.909523+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535459,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":774,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'4444","reported_seq":4567,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.915508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:24.915508+0000","last_peered":"2024-09-11T01:15:24.915508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:24.915508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4444,"log_dups_size":0,"ondisk_log_size":4444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18179014,"num_objects":4440,"num_object_clones":0,"num_object_copies":8880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4440,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":157},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4286","reported_seq":4395,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.861165+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:24.861165+0000","last_peered":"2024-09-11T01:15:24.861165+0000","last_clean":"2024-09-11T01:15:24.861165+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:24.861165+0000","last_undegraded":"2024-09-11T01:15:24.861165+0000","last_fullsized":"2024-09-11T01:15:24.861165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17547264,"num_objects":4284,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1239,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4283","reported_seq":4546,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:24.589711+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:24.589711+0000","last_peered":"2024-09-11T01:15:24.589711+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:24.589711+0000","last_undegraded":"2024-09-11T01:15:24.589711+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17514979,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4029,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":389120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4029},{"shards":"2","objects":248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4287","reported_seq":4406,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.912103+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:24.912103+0000","last_peered":"2024-09-11T01:15:24.912103+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:24.912103+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":2604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4189","reported_seq":4273,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.548223+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:24.548223+0000","last_peered":"2024-09-11T01:15:24.548223+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:24.548223+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4189,"log_dups_size":0,"ondisk_log_size":4189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17138147,"num_objects":4185,"num_object_clones":0,"num_object_copies":8370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1958,"num_write_kb":7832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":158}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4342","reported_seq":4425,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.552485+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:24.552485+0000","last_peered":"2024-09-11T01:15:24.552485+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:24.552485+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4342,"log_dups_size":0,"ondisk_log_size":4342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1979,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4328","reported_seq":4447,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.898009+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:24.898009+0000","last_peered":"2024-09-11T01:15:24.898009+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:24.898009+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17719296,"num_objects":4326,"num_object_clones":0,"num_object_copies":8652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1018,"num_write_kb":4072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4374","reported_seq":4498,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.894803+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:24.894803+0000","last_peered":"2024-09-11T01:15:24.894803+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:24.894803+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4311","reported_seq":4434,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.888612+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:24.888612+0000","last_peered":"2024-09-11T01:15:24.888612+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:24.888612+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4311,"log_dups_size":0,"ondisk_log_size":4311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641472,"num_objects":4307,"num_object_clones":0,"num_object_copies":8614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":3384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":159},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4322","reported_seq":4447,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.882305+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:24.882305+0000","last_peered":"2024-09-11T01:15:24.882305+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:24.882305+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4322,"log_dups_size":0,"ondisk_log_size":4322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17654726,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":979,"num_write_kb":3916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":158},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280990430,"num_objects":68610,"num_object_clones":0,"num_object_copies":137220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":4029,"num_objects_unfound":0,"num_objects_dirty":68610,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":27745,"num_write_kb":110980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":393216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":566894592,"data_stored":566739233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68682,"ondisk_log_size":68682,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404712,"kb_used_data":268888,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93967128,"statfs":{"total":96636764160,"available":96222339072,"internally_reserved":0,"allocated":275341312,"data_stored":275084408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":139062958},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84296,"kb_used_data":12344,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94287544,"statfs":{"total":96636764160,"available":96550445056,"internally_reserved":0,"allocated":12640256,"data_stored":12332306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411800,"kb_used_data":271880,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93960040,"statfs":{"total":96636764160,"available":96215080960,"internally_reserved":0,"allocated":278405120,"data_stored":278144139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,7,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278605824,"data_stored":278569694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12636160,"data_stored":12553061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275652608,"data_stored":275616478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:29.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:30.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 54 20 71860 71840 5.19629 1.94531 0.0324635 0.0150249 2024-09-11T01:15:30.323 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:30.324 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:30.354 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-11T01:15:28.422838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280990449,"num_objects":68612,"num_object_clones":0,"num_object_copies":137224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":4029,"num_objects_unfound":0,"num_objects_dirty":68612,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":27747,"num_write_kb":110982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":97,"num_bytes_recovered":393235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68685,"ondisk_log_size":68685,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":900808,"kb_used_data":553112,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282214712,"statfs":{"total":289910292480,"available":288987865088,"internally_reserved":0,"allocated":566386688,"data_stored":565560853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":355978247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,10,11,13,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25649152,"num_objects":6262,"num_object_clones":0,"num_object_copies":12524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":-42867,"num_objects_unfound":0,"num_objects_dirty":6262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6262,"num_write_kb":25048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":389120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.455685"},"pg_stats":[{"pgid":"2.f","version":"39'4270","reported_seq":4333,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.940101+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:24.940101+0000","last_peered":"2024-09-11T01:15:24.940101+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:24.940101+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4270,"log_dups_size":0,"ondisk_log_size":4270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4270,"num_write_kb":17080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":147},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4270","reported_seq":4340,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.607874+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:24.607874+0000","last_peered":"2024-09-11T01:15:24.607874+0000","last_clean":"2024-09-11T01:15:24.607874+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:24.607874+0000","last_undegraded":"2024-09-11T01:15:24.607874+0000","last_fullsized":"2024-09-11T01:15:24.607874+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4270,"log_dups_size":0,"ondisk_log_size":4270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17466310,"num_objects":4266,"num_object_clones":0,"num_object_copies":8532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4266,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2368,"num_write_kb":9472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4161","reported_seq":4233,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.576637+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:24.576637+0000","last_peered":"2024-09-11T01:15:24.576637+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:24.576637+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4161,"log_dups_size":0,"ondisk_log_size":4161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17027072,"num_objects":4157,"num_object_clones":0,"num_object_copies":8314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2222,"num_write_kb":8888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":152}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4237","reported_seq":4272,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.606607+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:24.606607+0000","last_peered":"2024-09-11T01:15:24.606607+0000","last_clean":"2024-09-11T01:15:24.606607+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:24.606607+0000","last_undegraded":"2024-09-11T01:15:24.606607+0000","last_fullsized":"2024-09-11T01:15:24.606607+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4237,"log_dups_size":0,"ondisk_log_size":4237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17354752,"num_objects":4237,"num_object_clones":0,"num_object_copies":8474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4292","reported_seq":4403,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.929050+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:24.929050+0000","last_peered":"2024-09-11T01:15:24.929050+0000","last_clean":"2024-09-11T01:15:24.929050+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:24.929050+0000","last_undegraded":"2024-09-11T01:15:24.929050+0000","last_fullsized":"2024-09-11T01:15:24.929050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4292,"log_dups_size":0,"ondisk_log_size":4292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17551843,"num_objects":4286,"num_object_clones":0,"num_object_copies":8572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1090,"num_write_kb":4360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4286","reported_seq":4410,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.909523+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:24.909523+0000","last_peered":"2024-09-11T01:15:24.909523+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:24.909523+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535459,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":774,"num_write_kb":3096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'4444","reported_seq":4567,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.915508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:24.915508+0000","last_peered":"2024-09-11T01:15:24.915508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:24.915508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4444,"log_dups_size":0,"ondisk_log_size":4444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18179014,"num_objects":4440,"num_object_clones":0,"num_object_copies":8880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4440,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1197,"num_write_kb":4788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":157},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4286","reported_seq":4395,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.861165+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:24.861165+0000","last_peered":"2024-09-11T01:15:24.861165+0000","last_clean":"2024-09-11T01:15:24.861165+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:24.861165+0000","last_undegraded":"2024-09-11T01:15:24.861165+0000","last_fullsized":"2024-09-11T01:15:24.861165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4286,"log_dups_size":0,"ondisk_log_size":4286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17547264,"num_objects":4284,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1239,"num_write_kb":4956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4283","reported_seq":4546,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:24.589711+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:24.589711+0000","last_peered":"2024-09-11T01:15:24.589711+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:24.589711+0000","last_undegraded":"2024-09-11T01:15:24.589711+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4283,"log_dups_size":0,"ondisk_log_size":4283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17514979,"num_objects":4277,"num_object_clones":0,"num_object_copies":8554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4029,"num_objects_unfound":0,"num_objects_dirty":4277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":95,"num_bytes_recovered":389120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4029},{"shards":"2","objects":248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4287","reported_seq":4406,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.912103+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:24.912103+0000","last_peered":"2024-09-11T01:15:24.912103+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:24.912103+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17526784,"num_objects":4279,"num_object_clones":0,"num_object_copies":8558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":651,"num_write_kb":2604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4189","reported_seq":4273,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.548223+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:24.548223+0000","last_peered":"2024-09-11T01:15:24.548223+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:24.548223+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4189,"log_dups_size":0,"ondisk_log_size":4189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17138147,"num_objects":4185,"num_object_clones":0,"num_object_copies":8370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1958,"num_write_kb":7832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":158}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4342","reported_seq":4425,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.552485+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:24.552485+0000","last_peered":"2024-09-11T01:15:24.552485+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:24.552485+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4342,"log_dups_size":0,"ondisk_log_size":4342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1979,"num_write_kb":7916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":159}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4328","reported_seq":4447,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.898009+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:24.898009+0000","last_peered":"2024-09-11T01:15:24.898009+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:24.898009+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17719296,"num_objects":4326,"num_object_clones":0,"num_object_copies":8652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1018,"num_write_kb":4072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":162},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4374","reported_seq":4498,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.894803+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:24.894803+0000","last_peered":"2024-09-11T01:15:24.894803+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:24.894803+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":767,"num_write_kb":3068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4311","reported_seq":4434,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.888612+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:24.888612+0000","last_peered":"2024-09-11T01:15:24.888612+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:24.888612+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4311,"log_dups_size":0,"ondisk_log_size":4311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641472,"num_objects":4307,"num_object_clones":0,"num_object_copies":8614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":846,"num_write_kb":3384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":159},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4322","reported_seq":4447,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.882305+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:24.882305+0000","last_peered":"2024-09-11T01:15:24.882305+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:24.882305+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4322,"log_dups_size":0,"ondisk_log_size":4322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17654726,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":979,"num_write_kb":3916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":158},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280990430,"num_objects":68610,"num_object_clones":0,"num_object_copies":137220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":4029,"num_objects_unfound":0,"num_objects_dirty":68610,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":27745,"num_write_kb":110980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":96,"num_bytes_recovered":393216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":566894592,"data_stored":566739233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68682,"ondisk_log_size":68682,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404712,"kb_used_data":268888,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93967128,"statfs":{"total":96636764160,"available":96222339072,"internally_reserved":0,"allocated":275341312,"data_stored":275084408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":139062958},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84296,"kb_used_data":12344,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94287544,"statfs":{"total":96636764160,"available":96550445056,"internally_reserved":0,"allocated":12640256,"data_stored":12332306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411800,"kb_used_data":271880,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93960040,"statfs":{"total":96636764160,"available":96215080960,"internally_reserved":0,"allocated":278405120,"data_stored":278144139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,7,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278605824,"data_stored":278569694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12636160,"data_stored":12553061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":275652608,"data_stored":275616478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:30.356 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:30.356 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'4270", 'reported_seq': 4333, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.940101+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:15:24.940101+0000', 'last_peered': '2024-09-11T01:15:24.940101+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:15:24.940101+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 4270, 'log_dups_size': 0, 'ondisk_log_size': 4270, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17489920, 'num_objects': 4270, 'num_object_clones': 0, 'num_object_copies': 8540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4270, 'num_write_kb': 17080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 147}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:30.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'4161", 'reported_seq': 4233, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.576637+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:24.576637+0000', 'last_peered': '2024-09-11T01:15:24.576637+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:24.576637+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 4161, 'log_dups_size': 0, 'ondisk_log_size': 4161, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17027072, 'num_objects': 4157, 'num_object_clones': 0, 'num_object_copies': 8314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4157, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2222, 'num_write_kb': 8888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 152}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:30.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'4286", 'reported_seq': 4410, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.909523+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:24.909523+0000', 'last_peered': '2024-09-11T01:15:24.909523+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:24.909523+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 4286, 'log_dups_size': 0, 'ondisk_log_size': 4286, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17535459, 'num_objects': 4282, 'num_object_clones': 0, 'num_object_copies': 8564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4282, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 774, 'num_write_kb': 3096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 167}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:30.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'4444", 'reported_seq': 4567, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.915508+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:15:24.915508+0000', 'last_peered': '2024-09-11T01:15:24.915508+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:15:24.915508+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 4444, 'log_dups_size': 0, 'ondisk_log_size': 4444, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18179014, 'num_objects': 4440, 'num_object_clones': 0, 'num_object_copies': 8880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4440, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1197, 'num_write_kb': 4788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 157}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:30.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'4283", 'reported_seq': 4546, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:24.589711+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:24.589711+0000', 'last_peered': '2024-09-11T01:15:24.589711+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:24.589711+0000', 'last_undegraded': '2024-09-11T01:15:24.589711+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 4283, 'log_dups_size': 0, 'ondisk_log_size': 4283, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17514979, 'num_objects': 4277, 'num_object_clones': 0, 'num_object_copies': 8554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4029, 'num_objects_unfound': 0, 'num_objects_dirty': 4277, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2150, 'num_write_kb': 8600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 95, 'num_bytes_recovered': 389120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4029}, {'shards': '2', 'objects': 248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:30.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'4287", 'reported_seq': 4406, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.912103+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:15:24.912103+0000', 'last_peered': '2024-09-11T01:15:24.912103+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:15:24.912103+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 4287, 'log_dups_size': 0, 'ondisk_log_size': 4287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17526784, 'num_objects': 4279, 'num_object_clones': 0, 'num_object_copies': 8558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 651, 'num_write_kb': 2604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 174}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:30.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'4189", 'reported_seq': 4273, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.548223+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:24.548223+0000', 'last_peered': '2024-09-11T01:15:24.548223+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:24.548223+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 4189, 'log_dups_size': 0, 'ondisk_log_size': 4189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17138147, 'num_objects': 4185, 'num_object_clones': 0, 'num_object_copies': 8370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4185, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1958, 'num_write_kb': 7832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 158}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:30.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'4342", 'reported_seq': 4425, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.552485+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:24.552485+0000', 'last_peered': '2024-09-11T01:15:24.552485+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:24.552485+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 4342, 'log_dups_size': 0, 'ondisk_log_size': 4342, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17760256, 'num_objects': 4336, 'num_object_clones': 0, 'num_object_copies': 8672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1979, 'num_write_kb': 7916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 159}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:30.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'4328", 'reported_seq': 4447, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.898009+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:15:24.898009+0000', 'last_peered': '2024-09-11T01:15:24.898009+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:15:24.898009+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 4328, 'log_dups_size': 0, 'ondisk_log_size': 4328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17719296, 'num_objects': 4326, 'num_object_clones': 0, 'num_object_copies': 8652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4326, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1018, 'num_write_kb': 4072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 162}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:30.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'4374", 'reported_seq': 4498, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.894803+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:15:24.894803+0000', 'last_peered': '2024-09-11T01:15:24.894803+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:15:24.894803+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 4374, 'log_dups_size': 0, 'ondisk_log_size': 4374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17883136, 'num_objects': 4366, 'num_object_clones': 0, 'num_object_copies': 8732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 767, 'num_write_kb': 3068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 174}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.361 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:30.361 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'4311", 'reported_seq': 4434, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.888612+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:24.888612+0000', 'last_peered': '2024-09-11T01:15:24.888612+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:24.888612+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 4311, 'log_dups_size': 0, 'ondisk_log_size': 4311, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17641472, 'num_objects': 4307, 'num_object_clones': 0, 'num_object_copies': 8614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 846, 'num_write_kb': 3384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 159}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.362 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:30.362 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'4322", 'reported_seq': 4447, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.882305+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:15:24.882305+0000', 'last_peered': '2024-09-11T01:15:24.882305+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:15:24.882305+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 4322, 'log_dups_size': 0, 'ondisk_log_size': 4322, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17654726, 'num_objects': 4312, 'num_object_clones': 0, 'num_object_copies': 8624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4312, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 979, 'num_write_kb': 3916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 158}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.363 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:31.071 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:31.071 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/143278 objects degraded (31.762%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":131,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":71639,"data_bytes":293389041,"bytes_used":946536448,"bytes_avail":288963756032,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":143278,"degraded_ratio":0.31762029062382224,"misplaced_objects":3902,"misplaced_total":143278,"misplaced_ratio":0.027233769315596248,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87102,"recovering_keys_per_sec":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"write_bytes_sec":2214063,"read_op_per_sec":0,"write_op_per_sec":540},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:31.100 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:31.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:31.316 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 55 20 72413 72393 5.14108 2.16016 0.0326872 0.0151876 2024-09-11T01:15:31.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:31.668 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:31.693 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2024-09-11T01:15:30.423271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293389041,"num_objects":71639,"num_object_clones":0,"num_object_copies":143278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":71639,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":30774,"num_write_kb":123090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":913427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71712,"ondisk_log_size":71712,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":924352,"kb_used_data":568336,"kb_used_omap":12,"kb_used_meta":355955,"kb_avail":282191168,"statfs":{"total":289910292480,"available":288963756032,"internally_reserved":0,"allocated":581976064,"data_stored":581150229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":364497927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,6,9,13,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3901,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.439504"},"pg_stats":[{"pgid":"2.f","version":"39'4450","reported_seq":4513,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.799086+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:29.799086+0000","last_peered":"2024-09-11T01:15:29.799086+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:29.799086+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4450,"log_dups_size":0,"ondisk_log_size":4450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":17800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":327},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4460","reported_seq":4530,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.655030+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:29.655030+0000","last_peered":"2024-09-11T01:15:29.655030+0000","last_clean":"2024-09-11T01:15:29.655030+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:29.655030+0000","last_undegraded":"2024-09-11T01:15:29.655030+0000","last_fullsized":"2024-09-11T01:15:29.655030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18244550,"num_objects":4456,"num_object_clones":0,"num_object_copies":8912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4350","reported_seq":4423,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.634210+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:29.634210+0000","last_peered":"2024-09-11T01:15:29.634210+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:29.634210+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2411,"num_write_kb":9644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":341}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4420","reported_seq":4455,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.621892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:29.621892+0000","last_peered":"2024-09-11T01:15:29.621892+0000","last_clean":"2024-09-11T01:15:29.621892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:29.621892+0000","last_undegraded":"2024-09-11T01:15:29.621892+0000","last_fullsized":"2024-09-11T01:15:29.621892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4420,"log_dups_size":0,"ondisk_log_size":4420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":17680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4482","reported_seq":4593,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.775670+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:29.775670+0000","last_peered":"2024-09-11T01:15:29.775670+0000","last_clean":"2024-09-11T01:15:29.775670+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:29.775670+0000","last_undegraded":"2024-09-11T01:15:29.775670+0000","last_fullsized":"2024-09-11T01:15:29.775670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18330083,"num_objects":4476,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1280,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4478","reported_seq":4602,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.895530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:29.895530+0000","last_peered":"2024-09-11T01:15:29.895530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:29.895530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4478,"log_dups_size":0,"ondisk_log_size":4478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321891,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":966,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":359},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'4654","reported_seq":4777,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.871340+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:29.871340+0000","last_peered":"2024-09-11T01:15:29.871340+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:29.871340+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19039174,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1407,"num_write_kb":5628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":367},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4462","reported_seq":4571,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.883227+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:29.883227+0000","last_peered":"2024-09-11T01:15:29.883227+0000","last_clean":"2024-09-11T01:15:29.883227+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:29.883227+0000","last_undegraded":"2024-09-11T01:15:29.883227+0000","last_fullsized":"2024-09-11T01:15:29.883227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4465","reported_seq":4982,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:29.629969+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:29.629969+0000","last_peered":"2024-09-11T01:15:29.629969+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:29.629969+0000","last_undegraded":"2024-09-11T01:15:29.629969+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4465,"log_dups_size":0,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18260451,"num_objects":4459,"num_object_clones":0,"num_object_copies":8918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":4459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2332,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3902},{"shards":"2","objects":557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4485","reported_seq":4604,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.876529+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:29.876529+0000","last_peered":"2024-09-11T01:15:29.876529+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:29.876529+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4485,"log_dups_size":0,"ondisk_log_size":4485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":372},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4373","reported_seq":4457,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.496011+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:29.496011+0000","last_peered":"2024-09-11T01:15:29.496011+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:29.496011+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891811,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2142,"num_write_kb":8568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":342}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4538","reported_seq":4619,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.596480+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:29.596480+0000","last_peered":"2024-09-11T01:15:29.596480+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:29.596480+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4538,"log_dups_size":0,"ondisk_log_size":4538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18563072,"num_objects":4532,"num_object_clones":0,"num_object_copies":9064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2175,"num_write_kb":8700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4504","reported_seq":4624,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.804626+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:29.804626+0000","last_peered":"2024-09-11T01:15:29.804626+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:29.804626+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4504,"log_dups_size":0,"ondisk_log_size":4504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":338},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4566","reported_seq":4690,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.885857+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:29.885857+0000","last_peered":"2024-09-11T01:15:29.885857+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:29.885857+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4566,"log_dups_size":0,"ondisk_log_size":4566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18669568,"num_objects":4558,"num_object_clones":0,"num_object_copies":9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":959,"num_write_kb":3836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4525","reported_seq":4648,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.889880+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:29.889880+0000","last_peered":"2024-09-11T01:15:29.889880+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:29.889880+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4525,"log_dups_size":0,"ondisk_log_size":4525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1060,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4497","reported_seq":4622,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.811475+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:29.811475+0000","last_peered":"2024-09-11T01:15:29.811475+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:29.811475+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371526,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1154,"num_write_kb":4616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":333},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293389022,"num_objects":71637,"num_object_clones":0,"num_object_copies":143274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":71637,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":30772,"num_write_kb":123088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":582250496,"data_stored":582095137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71709,"ondisk_log_size":71709,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414844,"kb_used_data":274860,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93956996,"statfs":{"total":96636764160,"available":96211963904,"internally_reserved":0,"allocated":281456640,"data_stored":281199736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84296,"kb_used_data":12344,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94287544,"statfs":{"total":96636764160,"available":96550445056,"internally_reserved":0,"allocated":12640256,"data_stored":12332306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425212,"kb_used_data":281132,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93946628,"statfs":{"total":96636764160,"available":96201347072,"internally_reserved":0,"allocated":287879168,"data_stored":287618187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287834112,"data_stored":287797982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12636160,"data_stored":12553061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":281780224,"data_stored":281744094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:32.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 56 20 73017 72997 5.09141 2.35938 0.0288236 0.0153363 2024-09-11T01:15:33.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 57 20 73650 73630 5.04546 2.47266 0.0147162 0.0154764 2024-09-11T01:15:34.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 58 19 74278 74259 5.00083 2.45703 0.0360088 0.0156139 2024-09-11T01:15:34.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:35.266 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:35.266 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:35.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-11T01:15:34.424219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293389041,"num_objects":71639,"num_object_clones":0,"num_object_copies":143278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":71639,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":30774,"num_write_kb":123090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":913427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71712,"ondisk_log_size":71712,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":925564,"kb_used_data":577868,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282189956,"statfs":{"total":289910292480,"available":288962514944,"internally_reserved":0,"allocated":591736832,"data_stored":590910997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":355978247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,10,13,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005221"},"pg_stats":[{"pgid":"2.f","version":"39'4450","reported_seq":4513,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.799086+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:29.799086+0000","last_peered":"2024-09-11T01:15:29.799086+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:29.799086+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4450,"log_dups_size":0,"ondisk_log_size":4450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":17800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":327},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4460","reported_seq":4530,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.655030+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:29.655030+0000","last_peered":"2024-09-11T01:15:29.655030+0000","last_clean":"2024-09-11T01:15:29.655030+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:29.655030+0000","last_undegraded":"2024-09-11T01:15:29.655030+0000","last_fullsized":"2024-09-11T01:15:29.655030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18244550,"num_objects":4456,"num_object_clones":0,"num_object_copies":8912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4350","reported_seq":4423,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.634210+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:29.634210+0000","last_peered":"2024-09-11T01:15:29.634210+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:29.634210+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2411,"num_write_kb":9644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":341}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4420","reported_seq":4455,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.621892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:29.621892+0000","last_peered":"2024-09-11T01:15:29.621892+0000","last_clean":"2024-09-11T01:15:29.621892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:29.621892+0000","last_undegraded":"2024-09-11T01:15:29.621892+0000","last_fullsized":"2024-09-11T01:15:29.621892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4420,"log_dups_size":0,"ondisk_log_size":4420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":17680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4482","reported_seq":4593,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.775670+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:29.775670+0000","last_peered":"2024-09-11T01:15:29.775670+0000","last_clean":"2024-09-11T01:15:29.775670+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:29.775670+0000","last_undegraded":"2024-09-11T01:15:29.775670+0000","last_fullsized":"2024-09-11T01:15:29.775670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18330083,"num_objects":4476,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1280,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4478","reported_seq":4602,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.895530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:29.895530+0000","last_peered":"2024-09-11T01:15:29.895530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:29.895530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4478,"log_dups_size":0,"ondisk_log_size":4478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321891,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":966,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":359},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'4654","reported_seq":4777,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.871340+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:29.871340+0000","last_peered":"2024-09-11T01:15:29.871340+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:29.871340+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19039174,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1407,"num_write_kb":5628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":367},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4462","reported_seq":4571,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.883227+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:29.883227+0000","last_peered":"2024-09-11T01:15:29.883227+0000","last_clean":"2024-09-11T01:15:29.883227+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:29.883227+0000","last_undegraded":"2024-09-11T01:15:29.883227+0000","last_fullsized":"2024-09-11T01:15:29.883227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4465","reported_seq":4982,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:29.629969+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:29.629969+0000","last_peered":"2024-09-11T01:15:29.629969+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:29.629969+0000","last_undegraded":"2024-09-11T01:15:29.629969+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4465,"log_dups_size":0,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18260451,"num_objects":4459,"num_object_clones":0,"num_object_copies":8918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":4459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2332,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3902},{"shards":"2","objects":557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4485","reported_seq":4604,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.876529+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:29.876529+0000","last_peered":"2024-09-11T01:15:29.876529+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:29.876529+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4485,"log_dups_size":0,"ondisk_log_size":4485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":372},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4373","reported_seq":4457,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.496011+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:29.496011+0000","last_peered":"2024-09-11T01:15:29.496011+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:29.496011+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891811,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2142,"num_write_kb":8568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":342}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4538","reported_seq":4619,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.596480+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:29.596480+0000","last_peered":"2024-09-11T01:15:29.596480+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:29.596480+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4538,"log_dups_size":0,"ondisk_log_size":4538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18563072,"num_objects":4532,"num_object_clones":0,"num_object_copies":9064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2175,"num_write_kb":8700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4504","reported_seq":4624,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.804626+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:29.804626+0000","last_peered":"2024-09-11T01:15:29.804626+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:29.804626+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4504,"log_dups_size":0,"ondisk_log_size":4504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":338},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4566","reported_seq":4690,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.885857+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:29.885857+0000","last_peered":"2024-09-11T01:15:29.885857+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:29.885857+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4566,"log_dups_size":0,"ondisk_log_size":4566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18669568,"num_objects":4558,"num_object_clones":0,"num_object_copies":9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":959,"num_write_kb":3836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4525","reported_seq":4648,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.889880+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:29.889880+0000","last_peered":"2024-09-11T01:15:29.889880+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:29.889880+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4525,"log_dups_size":0,"ondisk_log_size":4525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1060,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4497","reported_seq":4622,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.811475+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:29.811475+0000","last_peered":"2024-09-11T01:15:29.811475+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:29.811475+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371526,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1154,"num_write_kb":4616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":333},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293389022,"num_objects":71637,"num_object_clones":0,"num_object_copies":143274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":71637,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":30772,"num_write_kb":123088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":591847424,"data_stored":591692065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71709,"ondisk_log_size":71709,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414844,"kb_used_data":274860,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93956996,"statfs":{"total":96636764160,"available":96211963904,"internally_reserved":0,"allocated":281456640,"data_stored":281199736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789956,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85508,"kb_used_data":21876,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94286332,"statfs":{"total":96636764160,"available":96549203968,"internally_reserved":0,"allocated":22401024,"data_stored":22093074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425212,"kb_used_data":281132,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93946628,"statfs":{"total":96636764160,"available":96201347072,"internally_reserved":0,"allocated":287879168,"data_stored":287618187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287834112,"data_stored":287797982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22233088,"data_stored":22149989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":281780224,"data_stored":281744094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:35.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:35.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 59 20 74863 74843 4.95473 2.28125 0.0219483 0.0157599 2024-09-11T01:15:35.865 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:35.865 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:35.892 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-11T01:15:34.424219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293389041,"num_objects":71639,"num_object_clones":0,"num_object_copies":143278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":71639,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":30774,"num_write_kb":123090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":913427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71712,"ondisk_log_size":71712,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":925564,"kb_used_data":577868,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282189956,"statfs":{"total":289910292480,"available":288962514944,"internally_reserved":0,"allocated":591736832,"data_stored":590910997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":355978247},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,10,13,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5643,"num_write_kb":22572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005221"},"pg_stats":[{"pgid":"2.f","version":"39'4450","reported_seq":4513,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.799086+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:29.799086+0000","last_peered":"2024-09-11T01:15:29.799086+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:29.799086+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4450,"log_dups_size":0,"ondisk_log_size":4450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18227200,"num_objects":4450,"num_object_clones":0,"num_object_copies":8900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":17800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":327},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4460","reported_seq":4530,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.655030+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:29.655030+0000","last_peered":"2024-09-11T01:15:29.655030+0000","last_clean":"2024-09-11T01:15:29.655030+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:29.655030+0000","last_undegraded":"2024-09-11T01:15:29.655030+0000","last_fullsized":"2024-09-11T01:15:29.655030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18244550,"num_objects":4456,"num_object_clones":0,"num_object_copies":8912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2558,"num_write_kb":10232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4350","reported_seq":4423,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.634210+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:29.634210+0000","last_peered":"2024-09-11T01:15:29.634210+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:29.634210+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4350,"log_dups_size":0,"ondisk_log_size":4350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2411,"num_write_kb":9644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":341}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4420","reported_seq":4455,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.621892+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:29.621892+0000","last_peered":"2024-09-11T01:15:29.621892+0000","last_clean":"2024-09-11T01:15:29.621892+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:29.621892+0000","last_undegraded":"2024-09-11T01:15:29.621892+0000","last_fullsized":"2024-09-11T01:15:29.621892+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4420,"log_dups_size":0,"ondisk_log_size":4420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4420,"num_write_kb":17680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4482","reported_seq":4593,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.775670+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:29.775670+0000","last_peered":"2024-09-11T01:15:29.775670+0000","last_clean":"2024-09-11T01:15:29.775670+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:29.775670+0000","last_undegraded":"2024-09-11T01:15:29.775670+0000","last_fullsized":"2024-09-11T01:15:29.775670+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18330083,"num_objects":4476,"num_object_clones":0,"num_object_copies":8952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1280,"num_write_kb":5120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4478","reported_seq":4602,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.895530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:29.895530+0000","last_peered":"2024-09-11T01:15:29.895530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:29.895530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4478,"log_dups_size":0,"ondisk_log_size":4478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321891,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":966,"num_write_kb":3864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":359},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'4654","reported_seq":4777,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.871340+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:29.871340+0000","last_peered":"2024-09-11T01:15:29.871340+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:29.871340+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4654,"log_dups_size":0,"ondisk_log_size":4654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19039174,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1407,"num_write_kb":5628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":367},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4462","reported_seq":4571,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.883227+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:29.883227+0000","last_peered":"2024-09-11T01:15:29.883227+0000","last_clean":"2024-09-11T01:15:29.883227+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:29.883227+0000","last_undegraded":"2024-09-11T01:15:29.883227+0000","last_fullsized":"2024-09-11T01:15:29.883227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4462,"log_dups_size":0,"ondisk_log_size":4462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268160,"num_objects":4460,"num_object_clones":0,"num_object_copies":8920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1415,"num_write_kb":5660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4465","reported_seq":4982,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:29.629969+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:29.629969+0000","last_peered":"2024-09-11T01:15:29.629969+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:29.629969+0000","last_undegraded":"2024-09-11T01:15:29.629969+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4465,"log_dups_size":0,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18260451,"num_objects":4459,"num_object_clones":0,"num_object_copies":8918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":4459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2332,"num_write_kb":9328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3902},{"shards":"2","objects":557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4485","reported_seq":4604,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.876529+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:29.876529+0000","last_peered":"2024-09-11T01:15:29.876529+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:29.876529+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4485,"log_dups_size":0,"ondisk_log_size":4485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18337792,"num_objects":4477,"num_object_clones":0,"num_object_copies":8954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":372},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4373","reported_seq":4457,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.496011+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:29.496011+0000","last_peered":"2024-09-11T01:15:29.496011+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:29.496011+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17891811,"num_objects":4369,"num_object_clones":0,"num_object_copies":8738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2142,"num_write_kb":8568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":342}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4538","reported_seq":4619,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.596480+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:29.596480+0000","last_peered":"2024-09-11T01:15:29.596480+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:29.596480+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4538,"log_dups_size":0,"ondisk_log_size":4538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18563072,"num_objects":4532,"num_object_clones":0,"num_object_copies":9064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2175,"num_write_kb":8700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":355}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4504","reported_seq":4624,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.804626+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:29.804626+0000","last_peered":"2024-09-11T01:15:29.804626+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:29.804626+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4504,"log_dups_size":0,"ondisk_log_size":4504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1194,"num_write_kb":4776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":338},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4566","reported_seq":4690,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.885857+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:29.885857+0000","last_peered":"2024-09-11T01:15:29.885857+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:29.885857+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4566,"log_dups_size":0,"ondisk_log_size":4566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18669568,"num_objects":4558,"num_object_clones":0,"num_object_copies":9116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":959,"num_write_kb":3836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4525","reported_seq":4648,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.889880+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:29.889880+0000","last_peered":"2024-09-11T01:15:29.889880+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:29.889880+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4525,"log_dups_size":0,"ondisk_log_size":4525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18518016,"num_objects":4521,"num_object_clones":0,"num_object_copies":9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1060,"num_write_kb":4240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4497","reported_seq":4622,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.811475+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:29.811475+0000","last_peered":"2024-09-11T01:15:29.811475+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:29.811475+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4497,"log_dups_size":0,"ondisk_log_size":4497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18371526,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1154,"num_write_kb":4616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":333},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293389022,"num_objects":71637,"num_object_clones":0,"num_object_copies":143274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3902,"num_objects_unfound":0,"num_objects_dirty":71637,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":30772,"num_write_kb":123088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":591847424,"data_stored":591692065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71709,"ondisk_log_size":71709,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414844,"kb_used_data":274860,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93956996,"statfs":{"total":96636764160,"available":96211963904,"internally_reserved":0,"allocated":281456640,"data_stored":281199736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789956,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85508,"kb_used_data":21876,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94286332,"statfs":{"total":96636764160,"available":96549203968,"internally_reserved":0,"allocated":22401024,"data_stored":22093074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425212,"kb_used_data":281132,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93946628,"statfs":{"total":96636764160,"available":96201347072,"internally_reserved":0,"allocated":287879168,"data_stored":287618187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":287834112,"data_stored":287797982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22233088,"data_stored":22149989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":281780224,"data_stored":281744094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:35.893 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:35.893 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'4450", 'reported_seq': 4513, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.799086+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:15:29.799086+0000', 'last_peered': '2024-09-11T01:15:29.799086+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:15:29.799086+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 4450, 'log_dups_size': 0, 'ondisk_log_size': 4450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18227200, 'num_objects': 4450, 'num_object_clones': 0, 'num_object_copies': 8900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4450, 'num_write_kb': 17800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 327}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.893 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:35.893 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'4350", 'reported_seq': 4423, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.634210+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:29.634210+0000', 'last_peered': '2024-09-11T01:15:29.634210+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:29.634210+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 4350, 'log_dups_size': 0, 'ondisk_log_size': 4350, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17801216, 'num_objects': 4346, 'num_object_clones': 0, 'num_object_copies': 8692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2411, 'num_write_kb': 9644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 341}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:35.893 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:35.893 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'4478", 'reported_seq': 4602, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.895530+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:29.895530+0000', 'last_peered': '2024-09-11T01:15:29.895530+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:29.895530+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 4478, 'log_dups_size': 0, 'ondisk_log_size': 4478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18321891, 'num_objects': 4474, 'num_object_clones': 0, 'num_object_copies': 8948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4474, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 966, 'num_write_kb': 3864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 359}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'4654", 'reported_seq': 4777, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.871340+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:15:29.871340+0000', 'last_peered': '2024-09-11T01:15:29.871340+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:15:29.871340+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 4654, 'log_dups_size': 0, 'ondisk_log_size': 4654, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19039174, 'num_objects': 4650, 'num_object_clones': 0, 'num_object_copies': 9300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4650, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1407, 'num_write_kb': 5628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 367}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'4465", 'reported_seq': 4982, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:29.629969+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:29.629969+0000', 'last_peered': '2024-09-11T01:15:29.629969+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:29.629969+0000', 'last_undegraded': '2024-09-11T01:15:29.629969+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 4465, 'log_dups_size': 0, 'ondisk_log_size': 4465, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18260451, 'num_objects': 4459, 'num_object_clones': 0, 'num_object_copies': 8918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3902, 'num_objects_unfound': 0, 'num_objects_dirty': 4459, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2332, 'num_write_kb': 9328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 222, 'num_bytes_recovered': 909312, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3902}, {'shards': '2', 'objects': 557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'4485", 'reported_seq': 4604, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.876529+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:15:29.876529+0000', 'last_peered': '2024-09-11T01:15:29.876529+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:15:29.876529+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 4485, 'log_dups_size': 0, 'ondisk_log_size': 4485, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18337792, 'num_objects': 4477, 'num_object_clones': 0, 'num_object_copies': 8954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 849, 'num_write_kb': 3396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 372}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'4373", 'reported_seq': 4457, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.496011+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:29.496011+0000', 'last_peered': '2024-09-11T01:15:29.496011+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:29.496011+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 4373, 'log_dups_size': 0, 'ondisk_log_size': 4373, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17891811, 'num_objects': 4369, 'num_object_clones': 0, 'num_object_copies': 8738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4369, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2142, 'num_write_kb': 8568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 342}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'4538", 'reported_seq': 4619, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.596480+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:29.596480+0000', 'last_peered': '2024-09-11T01:15:29.596480+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:29.596480+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 4538, 'log_dups_size': 0, 'ondisk_log_size': 4538, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18563072, 'num_objects': 4532, 'num_object_clones': 0, 'num_object_copies': 9064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2175, 'num_write_kb': 8700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 355}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:35.894 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'4504", 'reported_seq': 4624, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.804626+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:15:29.804626+0000', 'last_peered': '2024-09-11T01:15:29.804626+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:15:29.804626+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 4504, 'log_dups_size': 0, 'ondisk_log_size': 4504, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18440192, 'num_objects': 4502, 'num_object_clones': 0, 'num_object_copies': 9004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1194, 'num_write_kb': 4776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 338}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'4566", 'reported_seq': 4690, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.885857+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:15:29.885857+0000', 'last_peered': '2024-09-11T01:15:29.885857+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:15:29.885857+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 4566, 'log_dups_size': 0, 'ondisk_log_size': 4566, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18669568, 'num_objects': 4558, 'num_object_clones': 0, 'num_object_copies': 9116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 959, 'num_write_kb': 3836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 366}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'4525", 'reported_seq': 4648, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.889880+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:29.889880+0000', 'last_peered': '2024-09-11T01:15:29.889880+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:29.889880+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 4525, 'log_dups_size': 0, 'ondisk_log_size': 4525, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18518016, 'num_objects': 4521, 'num_object_clones': 0, 'num_object_copies': 9042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1060, 'num_write_kb': 4240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 373}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:35.895 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'4497", 'reported_seq': 4622, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:29.811475+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:15:29.811475+0000', 'last_peered': '2024-09-11T01:15:29.811475+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:15:29.811475+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 4497, 'log_dups_size': 0, 'ondisk_log_size': 4497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18371526, 'num_objects': 4487, 'num_object_clones': 0, 'num_object_copies': 8974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4487, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1154, 'num_write_kb': 4616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 333}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:35.895 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:36.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:15:36.316460+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0159097 2024-09-11T01:15:36.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:15:36.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 60 20 75409 75389 4.9077 2.13281 0.0756398 0.0159097 2024-09-11T01:15:36.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:36.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/149206 objects degraded (30.500%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74603,"data_bytes":305529585,"bytes_used":971411456,"bytes_avail":288938881024,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":149206,"degraded_ratio":0.30500113936436873,"misplaced_objects":3773,"misplaced_total":149206,"misplaced_ratio":0.025287186842352184,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":119781,"recovering_keys_per_sec":0,"num_objects_recovered":351,"num_bytes_recovered":1437696,"num_keys_recovered":0,"write_bytes_sec":2937205,"read_op_per_sec":0,"write_op_per_sec":717},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:36.679 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:36.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:37.257 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:37.259 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:37.285 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-11T01:15:36.424649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":305529585,"num_objects":74603,"num_object_clones":0,"num_object_copies":149206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3773,"num_objects_unfound":0,"num_objects_dirty":74603,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":33738,"num_write_kb":134946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":1441811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74676,"ondisk_log_size":74676,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":948644,"kb_used_data":592624,"kb_used_omap":12,"kb_used_meta":355955,"kb_avail":282166876,"statfs":{"total":289910292480,"available":288938881024,"internally_reserved":0,"allocated":606846976,"data_stored":606021141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":364497927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,5,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35254272,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3773,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8607,"num_write_kb":34428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002655"},"pg_stats":[{"pgid":"2.f","version":"39'4632","reported_seq":4695,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.873084+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:34.873084+0000","last_peered":"2024-09-11T01:15:34.873084+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:34.873084+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4632,"num_write_kb":18528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":509},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4638","reported_seq":4708,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.656815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:34.656815+0000","last_peered":"2024-09-11T01:15:34.656815+0000","last_clean":"2024-09-11T01:15:34.656815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:34.656815+0000","last_undegraded":"2024-09-11T01:15:34.656815+0000","last_fullsized":"2024-09-11T01:15:34.656815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18973638,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2736,"num_write_kb":10944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4519","reported_seq":4592,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.646016+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:34.646016+0000","last_peered":"2024-09-11T01:15:34.646016+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:34.646016+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18493440,"num_objects":4515,"num_object_clones":0,"num_object_copies":9030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2580,"num_write_kb":10320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4613","reported_seq":4648,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.584669+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:34.584669+0000","last_peered":"2024-09-11T01:15:34.584669+0000","last_clean":"2024-09-11T01:15:34.584669+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:34.584669+0000","last_undegraded":"2024-09-11T01:15:34.584669+0000","last_fullsized":"2024-09-11T01:15:34.584669+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4613,"log_dups_size":0,"ondisk_log_size":4613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18894848,"num_objects":4613,"num_object_clones":0,"num_object_copies":9226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4613,"num_write_kb":18452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4668","reported_seq":4779,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.923754+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:34.923754+0000","last_peered":"2024-09-11T01:15:34.923754+0000","last_clean":"2024-09-11T01:15:34.923754+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:34.923754+0000","last_undegraded":"2024-09-11T01:15:34.923754+0000","last_fullsized":"2024-09-11T01:15:34.923754+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4668,"log_dups_size":0,"ondisk_log_size":4668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19091939,"num_objects":4662,"num_object_clones":0,"num_object_copies":9324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1466,"num_write_kb":5864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4626","reported_seq":4750,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.927751+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:34.927751+0000","last_peered":"2024-09-11T01:15:34.927751+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:34.927751+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4626,"log_dups_size":0,"ondisk_log_size":4626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18928099,"num_objects":4622,"num_object_clones":0,"num_object_copies":9244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1114,"num_write_kb":4456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":507},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'4858","reported_seq":4976,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.944072+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:34.944072+0000","last_peered":"2024-09-11T01:15:34.944072+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:34.944072+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":4858,"log_dups_size":0,"ondisk_log_size":4858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19874758,"num_objects":4854,"num_object_clones":0,"num_object_copies":9708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4854,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1611,"num_write_kb":6444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4664","reported_seq":4773,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.884098+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:34.884098+0000","last_peered":"2024-09-11T01:15:34.884098+0000","last_clean":"2024-09-11T01:15:34.884098+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:34.884098+0000","last_undegraded":"2024-09-11T01:15:34.884098+0000","last_fullsized":"2024-09-11T01:15:34.884098+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4664,"log_dups_size":0,"ondisk_log_size":4664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19095552,"num_objects":4662,"num_object_clones":0,"num_object_copies":9324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1617,"num_write_kb":6468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4640","reported_seq":5413,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:34.653703+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:34.653703+0000","last_peered":"2024-09-11T01:15:34.653703+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:34.653703+0000","last_undegraded":"2024-09-11T01:15:34.653703+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18977251,"num_objects":4634,"num_object_clones":0,"num_object_copies":9268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3773,"num_objects_unfound":0,"num_objects_dirty":4634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3773},{"shards":"2","objects":861}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4665","reported_seq":4784,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.642488+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:34.642488+0000","last_peered":"2024-09-11T01:15:34.642488+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:34.642488+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1029,"num_write_kb":4116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":552},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4567","reported_seq":4648,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.637306+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:34.637306+0000","last_peered":"2024-09-11T01:15:34.637306+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:34.637306+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18686435,"num_objects":4563,"num_object_clones":0,"num_object_copies":9126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4563,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":536}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4729","reported_seq":4812,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.621790+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:34.621790+0000","last_peered":"2024-09-11T01:15:34.621790+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:34.621790+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4729,"log_dups_size":0,"ondisk_log_size":4729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19345408,"num_objects":4723,"num_object_clones":0,"num_object_copies":9446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2366,"num_write_kb":9464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":546}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4689","reported_seq":4807,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.934346+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:34.934346+0000","last_peered":"2024-09-11T01:15:34.934346+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:34.934346+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19197952,"num_objects":4687,"num_object_clones":0,"num_object_copies":9374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1379,"num_write_kb":5516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":523},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4769","reported_seq":4893,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.915173+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:34.915173+0000","last_peered":"2024-09-11T01:15:34.915173+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:34.915173+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4769,"log_dups_size":0,"ondisk_log_size":4769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501056,"num_objects":4761,"num_object_clones":0,"num_object_copies":9522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1162,"num_write_kb":4648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4735","reported_seq":4858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.950476+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:34.950476+0000","last_peered":"2024-09-11T01:15:34.950476+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:34.950476+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4735,"log_dups_size":0,"ondisk_log_size":4735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19378176,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1270,"num_write_kb":5080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":583},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4661","reported_seq":4786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.852775+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:34.852775+0000","last_peered":"2024-09-11T01:15:34.852775+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:34.852775+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4661,"log_dups_size":0,"ondisk_log_size":4661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19043270,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1318,"num_write_kb":5272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":497},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":305529566,"num_objects":74601,"num_object_clones":0,"num_object_copies":149202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3773,"num_objects_unfound":0,"num_objects_dirty":74601,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":33736,"num_write_kb":134944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":1441792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":607195136,"data_stored":607039777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74673,"ondisk_log_size":74673,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":424908,"kb_used_data":280764,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93946932,"statfs":{"total":96636764160,"available":96201658368,"internally_reserved":0,"allocated":287502336,"data_stored":287245432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":147582638},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789956,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":85508,"kb_used_data":21876,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94286332,"statfs":{"total":96636764160,"available":96549203968,"internally_reserved":0,"allocated":22401024,"data_stored":22093074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":438228,"kb_used_data":289984,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93933612,"statfs":{"total":96636764160,"available":96188018688,"internally_reserved":0,"allocated":296943616,"data_stored":296682635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":151776938},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":297091072,"data_stored":297054942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22233088,"data_stored":22149989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":287870976,"data_stored":287834846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:37.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 61 20 75958 75938 4.8624 2.14453 0.0127379 0.0160542 2024-09-11T01:15:38.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 62 20 76593 76573 4.82397 2.48047 0.020297 0.0161845 2024-09-11T01:15:39.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 63 20 77214 77194 4.7859 2.42578 0.0164727 0.0163139 2024-09-11T01:15:40.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:40.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 64 20 77826 77806 4.74847 2.39062 0.019826 0.0164455 2024-09-11T01:15:40.871 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:40.871 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:40.896 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-11T01:15:40.425580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317641457,"num_objects":77560,"num_object_clones":0,"num_object_copies":155120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77560,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36695,"num_write_kb":146774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":1949715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77633,"ondisk_log_size":77633,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1031040,"kb_used_data":616720,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282084480,"statfs":{"total":289910292480,"available":288854507520,"internally_reserved":0,"allocated":631521280,"data_stored":630695445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":424201223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,4,8,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36651008,"num_objects":8948,"num_object_clones":0,"num_object_copies":17896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":8948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":35792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002742"},"pg_stats":[{"pgid":"2.f","version":"39'4839","reported_seq":4902,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.974972+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:39.974972+0000","last_peered":"2024-09-11T01:15:39.974972+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:39.974972+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19820544,"num_objects":4839,"num_object_clones":0,"num_object_copies":9678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4839,"num_write_kb":19356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":716},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4847","reported_seq":4917,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.602011+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:39.602011+0000","last_peered":"2024-09-11T01:15:39.602011+0000","last_clean":"2024-09-11T01:15:39.602011+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:39.602011+0000","last_undegraded":"2024-09-11T01:15:39.602011+0000","last_fullsized":"2024-09-11T01:15:39.602011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19829702,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4714","reported_seq":4786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.698124+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:39.698124+0000","last_peered":"2024-09-11T01:15:39.698124+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:39.698124+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292160,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2775,"num_write_kb":11100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4811","reported_seq":4846,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.701982+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:39.701982+0000","last_peered":"2024-09-11T01:15:39.701982+0000","last_clean":"2024-09-11T01:15:39.701982+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:39.701982+0000","last_undegraded":"2024-09-11T01:15:39.701982+0000","last_fullsized":"2024-09-11T01:15:39.701982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4811,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4836","reported_seq":4947,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.919121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:39.919121+0000","last_peered":"2024-09-11T01:15:39.919121+0000","last_clean":"2024-09-11T01:15:39.919121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:39.919121+0000","last_undegraded":"2024-09-11T01:15:39.919121+0000","last_fullsized":"2024-09-11T01:15:39.919121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19780067,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1634,"num_write_kb":6536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4802","reported_seq":4926,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.918922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:39.918922+0000","last_peered":"2024-09-11T01:15:39.918922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:39.918922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648995,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1290,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5025","reported_seq":5148,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.956162+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:39.956162+0000","last_peered":"2024-09-11T01:15:39.956162+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:39.956162+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20558790,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1778,"num_write_kb":7112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4845","reported_seq":4954,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.954425+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:39.954425+0000","last_peered":"2024-09-11T01:15:39.954425+0000","last_clean":"2024-09-11T01:15:39.954425+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:39.954425+0000","last_undegraded":"2024-09-11T01:15:39.954425+0000","last_fullsized":"2024-09-11T01:15:39.954425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19836928,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1798,"num_write_kb":7192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4814","reported_seq":5836,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.712895+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:39.712895+0000","last_peered":"2024-09-11T01:15:39.712895+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:39.712895+0000","last_undegraded":"2024-09-11T01:15:39.712895+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4814,"log_dups_size":0,"ondisk_log_size":4814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689955,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":1945600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3649},{"shards":"2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4855","reported_seq":4974,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.921147+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:39.921147+0000","last_peered":"2024-09-11T01:15:39.921147+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:39.921147+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1219,"num_write_kb":4876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":742},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4751","reported_seq":4835,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.690037+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:39.690037+0000","last_peered":"2024-09-11T01:15:39.690037+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:39.690037+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440099,"num_objects":4747,"num_object_clones":0,"num_object_copies":9494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4898","reported_seq":4981,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.662739+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:39.662739+0000","last_peered":"2024-09-11T01:15:39.662739+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:39.662739+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20037632,"num_objects":4892,"num_object_clones":0,"num_object_copies":9784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4887","reported_seq":5007,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.997651+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:39.997651+0000","last_peered":"2024-09-11T01:15:39.997651+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:39.997651+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":721},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4953","reported_seq":5077,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.993093+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:39.993093+0000","last_peered":"2024-09-11T01:15:39.993093+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:39.993093+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4915","reported_seq":5038,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.982026+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:39.982026+0000","last_peered":"2024-09-11T01:15:39.982026+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:39.982026+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4915,"log_dups_size":0,"ondisk_log_size":4915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":763},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4838","reported_seq":4963,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.952973+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:39.952973+0000","last_peered":"2024-09-11T01:15:39.952973+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:39.952973+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19768262,"num_objects":4828,"num_object_clones":0,"num_object_copies":9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1495,"num_write_kb":5980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":674},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317641438,"num_objects":77558,"num_object_clones":0,"num_object_copies":155116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77558,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36693,"num_write_kb":146772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":1949696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":631967744,"data_stored":631812385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77630,"ondisk_log_size":77630,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439172,"kb_used_data":286708,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93932668,"statfs":{"total":96636764160,"available":96187052032,"internally_reserved":0,"allocated":293588992,"data_stored":293332088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789957,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134956,"kb_used_data":31068,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94236884,"statfs":{"total":96636764160,"available":96498569216,"internally_reserved":0,"allocated":31813632,"data_stored":31505682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456912,"kb_used_data":298944,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93914928,"statfs":{"total":96636764160,"available":96168886272,"internally_reserved":0,"allocated":306118656,"data_stored":305857675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306241536,"data_stored":306205406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31793152,"data_stored":31710053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293933056,"data_stored":293896926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:40.898 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:41.317 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 65 20 78354 78334 4.70714 2.0625 0.0208215 0.0165887 2024-09-11T01:15:41.508 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:41.508 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:41.544 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-11T01:15:40.425580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317641457,"num_objects":77560,"num_object_clones":0,"num_object_copies":155120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77560,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36695,"num_write_kb":146774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":1949715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77633,"ondisk_log_size":77633,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1031040,"kb_used_data":616720,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282084480,"statfs":{"total":289910292480,"available":288854507520,"internally_reserved":0,"allocated":631521280,"data_stored":630695445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":424201223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,4,8,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36651008,"num_objects":8948,"num_object_clones":0,"num_object_copies":17896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-380,"num_objects_unfound":0,"num_objects_dirty":8948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":35792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002742"},"pg_stats":[{"pgid":"2.f","version":"39'4839","reported_seq":4902,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.974972+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:39.974972+0000","last_peered":"2024-09-11T01:15:39.974972+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:39.974972+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19820544,"num_objects":4839,"num_object_clones":0,"num_object_copies":9678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4839,"num_write_kb":19356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":716},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4847","reported_seq":4917,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.602011+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:39.602011+0000","last_peered":"2024-09-11T01:15:39.602011+0000","last_clean":"2024-09-11T01:15:39.602011+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:39.602011+0000","last_undegraded":"2024-09-11T01:15:39.602011+0000","last_fullsized":"2024-09-11T01:15:39.602011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19829702,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4714","reported_seq":4786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.698124+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:39.698124+0000","last_peered":"2024-09-11T01:15:39.698124+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:39.698124+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292160,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2775,"num_write_kb":11100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4811","reported_seq":4846,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.701982+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:39.701982+0000","last_peered":"2024-09-11T01:15:39.701982+0000","last_clean":"2024-09-11T01:15:39.701982+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:39.701982+0000","last_undegraded":"2024-09-11T01:15:39.701982+0000","last_fullsized":"2024-09-11T01:15:39.701982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4811,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4836","reported_seq":4947,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.919121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:39.919121+0000","last_peered":"2024-09-11T01:15:39.919121+0000","last_clean":"2024-09-11T01:15:39.919121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:39.919121+0000","last_undegraded":"2024-09-11T01:15:39.919121+0000","last_fullsized":"2024-09-11T01:15:39.919121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19780067,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1634,"num_write_kb":6536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4802","reported_seq":4926,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.918922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:39.918922+0000","last_peered":"2024-09-11T01:15:39.918922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:39.918922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648995,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1290,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5025","reported_seq":5148,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.956162+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:39.956162+0000","last_peered":"2024-09-11T01:15:39.956162+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:39.956162+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20558790,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1778,"num_write_kb":7112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4845","reported_seq":4954,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.954425+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:39.954425+0000","last_peered":"2024-09-11T01:15:39.954425+0000","last_clean":"2024-09-11T01:15:39.954425+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:39.954425+0000","last_undegraded":"2024-09-11T01:15:39.954425+0000","last_fullsized":"2024-09-11T01:15:39.954425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19836928,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1798,"num_write_kb":7192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4814","reported_seq":5836,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.712895+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:39.712895+0000","last_peered":"2024-09-11T01:15:39.712895+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:39.712895+0000","last_undegraded":"2024-09-11T01:15:39.712895+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4814,"log_dups_size":0,"ondisk_log_size":4814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689955,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":1945600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3649},{"shards":"2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4855","reported_seq":4974,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.921147+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:39.921147+0000","last_peered":"2024-09-11T01:15:39.921147+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:39.921147+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1219,"num_write_kb":4876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":742},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4751","reported_seq":4835,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.690037+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:39.690037+0000","last_peered":"2024-09-11T01:15:39.690037+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:39.690037+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440099,"num_objects":4747,"num_object_clones":0,"num_object_copies":9494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4898","reported_seq":4981,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.662739+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:39.662739+0000","last_peered":"2024-09-11T01:15:39.662739+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:39.662739+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20037632,"num_objects":4892,"num_object_clones":0,"num_object_copies":9784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4887","reported_seq":5007,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.997651+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:39.997651+0000","last_peered":"2024-09-11T01:15:39.997651+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:39.997651+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":721},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4953","reported_seq":5077,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.993093+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:39.993093+0000","last_peered":"2024-09-11T01:15:39.993093+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:39.993093+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4915","reported_seq":5038,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.982026+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:39.982026+0000","last_peered":"2024-09-11T01:15:39.982026+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:39.982026+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4915,"log_dups_size":0,"ondisk_log_size":4915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":763},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4838","reported_seq":4963,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.952973+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:39.952973+0000","last_peered":"2024-09-11T01:15:39.952973+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:39.952973+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19768262,"num_objects":4828,"num_object_clones":0,"num_object_copies":9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1495,"num_write_kb":5980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":674},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317641438,"num_objects":77558,"num_object_clones":0,"num_object_copies":155116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77558,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36693,"num_write_kb":146772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":1949696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":631967744,"data_stored":631812385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77630,"ondisk_log_size":77630,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439172,"kb_used_data":286708,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93932668,"statfs":{"total":96636764160,"available":96187052032,"internally_reserved":0,"allocated":293588992,"data_stored":293332088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789957,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":134956,"kb_used_data":31068,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94236884,"statfs":{"total":96636764160,"available":96498569216,"internally_reserved":0,"allocated":31813632,"data_stored":31505682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456912,"kb_used_data":298944,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93914928,"statfs":{"total":96636764160,"available":96168886272,"internally_reserved":0,"allocated":306118656,"data_stored":305857675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306241536,"data_stored":306205406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31793152,"data_stored":31710053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293933056,"data_stored":293896926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:41.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'4839", 'reported_seq': 4902, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.974972+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:15:39.974972+0000', 'last_peered': '2024-09-11T01:15:39.974972+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:15:39.974972+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 4839, 'log_dups_size': 0, 'ondisk_log_size': 4839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19820544, 'num_objects': 4839, 'num_object_clones': 0, 'num_object_copies': 9678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4839, 'num_write_kb': 19356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 716}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'4714", 'reported_seq': 4786, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.698124+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:39.698124+0000', 'last_peered': '2024-09-11T01:15:39.698124+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:39.698124+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 4714, 'log_dups_size': 0, 'ondisk_log_size': 4714, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19292160, 'num_objects': 4710, 'num_object_clones': 0, 'num_object_copies': 9420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2775, 'num_write_kb': 11100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'4802", 'reported_seq': 4926, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.918922+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:39.918922+0000', 'last_peered': '2024-09-11T01:15:39.918922+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:39.918922+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 4802, 'log_dups_size': 0, 'ondisk_log_size': 4802, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19648995, 'num_objects': 4798, 'num_object_clones': 0, 'num_object_copies': 9596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4798, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1290, 'num_write_kb': 5160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 683}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'5025", 'reported_seq': 5148, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.956162+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:15:39.956162+0000', 'last_peered': '2024-09-11T01:15:39.956162+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:15:39.956162+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 5025, 'log_dups_size': 0, 'ondisk_log_size': 5025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20558790, 'num_objects': 5021, 'num_object_clones': 0, 'num_object_copies': 10042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5021, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1778, 'num_write_kb': 7112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 738}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'4814", 'reported_seq': 5836, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:39.712895+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:39.712895+0000', 'last_peered': '2024-09-11T01:15:39.712895+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:39.712895+0000', 'last_undegraded': '2024-09-11T01:15:39.712895+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 4814, 'log_dups_size': 0, 'ondisk_log_size': 4814, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19689955, 'num_objects': 4808, 'num_object_clones': 0, 'num_object_copies': 9616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3649, 'num_objects_unfound': 0, 'num_objects_dirty': 4808, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2681, 'num_write_kb': 10724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 475, 'num_bytes_recovered': 1945600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3649}, {'shards': '2', 'objects': 1159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:41.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'4855", 'reported_seq': 4974, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.921147+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:15:39.921147+0000', 'last_peered': '2024-09-11T01:15:39.921147+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:15:39.921147+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 4855, 'log_dups_size': 0, 'ondisk_log_size': 4855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19853312, 'num_objects': 4847, 'num_object_clones': 0, 'num_object_copies': 9694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4847, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1219, 'num_write_kb': 4876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 742}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'4751", 'reported_seq': 4835, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.690037+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:39.690037+0000', 'last_peered': '2024-09-11T01:15:39.690037+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:39.690037+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 4751, 'log_dups_size': 0, 'ondisk_log_size': 4751, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19440099, 'num_objects': 4747, 'num_object_clones': 0, 'num_object_copies': 9494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4747, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2520, 'num_write_kb': 10080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 720}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'4898", 'reported_seq': 4981, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.662739+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:39.662739+0000', 'last_peered': '2024-09-11T01:15:39.662739+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:39.662739+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 4898, 'log_dups_size': 0, 'ondisk_log_size': 4898, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20037632, 'num_objects': 4892, 'num_object_clones': 0, 'num_object_copies': 9784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4892, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2535, 'num_write_kb': 10140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 715}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'4887", 'reported_seq': 5007, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.997651+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:15:39.997651+0000', 'last_peered': '2024-09-11T01:15:39.997651+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:15:39.997651+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 4887, 'log_dups_size': 0, 'ondisk_log_size': 4887, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20008960, 'num_objects': 4885, 'num_object_clones': 0, 'num_object_copies': 9770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1577, 'num_write_kb': 6308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 721}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'4953", 'reported_seq': 5077, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.993093+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:15:39.993093+0000', 'last_peered': '2024-09-11T01:15:39.993093+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:15:39.993093+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 4953, 'log_dups_size': 0, 'ondisk_log_size': 4953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20254720, 'num_objects': 4945, 'num_object_clones': 0, 'num_object_copies': 9890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1346, 'num_write_kb': 5384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 753}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:41.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'4915", 'reported_seq': 5038, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.982026+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:39.982026+0000', 'last_peered': '2024-09-11T01:15:39.982026+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:39.982026+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 4915, 'log_dups_size': 0, 'ondisk_log_size': 4915, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20115456, 'num_objects': 4911, 'num_object_clones': 0, 'num_object_copies': 9822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4911, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1450, 'num_write_kb': 5800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 763}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.548 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:41.548 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'4838", 'reported_seq': 4963, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.952973+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:15:39.952973+0000', 'last_peered': '2024-09-11T01:15:39.952973+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:15:39.952973+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 4838, 'log_dups_size': 0, 'ondisk_log_size': 4838, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19768262, 'num_objects': 4828, 'num_object_clones': 0, 'num_object_copies': 9656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4828, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1495, 'num_write_kb': 5980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 674}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:41.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:42.243 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:42.243 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/155120 objects degraded (29.337%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":142,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77560,"data_bytes":317641457,"bytes_used":1055784960,"bytes_avail":288854507520,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":155120,"degraded_ratio":0.29337287261474987,"misplaced_objects":3649,"misplaced_total":155120,"misplaced_ratio":0.023523723568849922,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":129677,"recovering_keys_per_sec":0,"num_objects_recovered":380,"num_bytes_recovered":1556480,"num_keys_recovered":0,"write_bytes_sec":3053552,"read_op_per_sec":0,"write_op_per_sec":745},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:42.270 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:42.270 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:42.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 66 20 78888 78868 4.66742 2.08594 0.0189592 0.0167307 2024-09-11T01:15:42.849 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:42.852 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:42.879 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-11T01:15:42.426097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317641457,"num_objects":77560,"num_object_clones":0,"num_object_copies":155120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77560,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36695,"num_write_kb":146774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":1949715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77633,"ondisk_log_size":77633,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1048684,"kb_used_data":626044,"kb_used_omap":12,"kb_used_meta":422579,"kb_avail":282066836,"statfs":{"total":289910292480,"available":288836440064,"internally_reserved":0,"allocated":641069056,"data_stored":640243221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":432720903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,1,5,11,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5921,"num_write_kb":23684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002826"},"pg_stats":[{"pgid":"2.f","version":"39'4839","reported_seq":4902,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.974972+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:39.974972+0000","last_peered":"2024-09-11T01:15:39.974972+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:39.974972+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19820544,"num_objects":4839,"num_object_clones":0,"num_object_copies":9678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4839,"num_write_kb":19356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":716},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4847","reported_seq":4917,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.602011+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:39.602011+0000","last_peered":"2024-09-11T01:15:39.602011+0000","last_clean":"2024-09-11T01:15:39.602011+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:39.602011+0000","last_undegraded":"2024-09-11T01:15:39.602011+0000","last_fullsized":"2024-09-11T01:15:39.602011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19829702,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4714","reported_seq":4786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.698124+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:39.698124+0000","last_peered":"2024-09-11T01:15:39.698124+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:39.698124+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292160,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2775,"num_write_kb":11100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4811","reported_seq":4846,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.701982+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:39.701982+0000","last_peered":"2024-09-11T01:15:39.701982+0000","last_clean":"2024-09-11T01:15:39.701982+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:39.701982+0000","last_undegraded":"2024-09-11T01:15:39.701982+0000","last_fullsized":"2024-09-11T01:15:39.701982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4811,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4836","reported_seq":4947,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.919121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:39.919121+0000","last_peered":"2024-09-11T01:15:39.919121+0000","last_clean":"2024-09-11T01:15:39.919121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:39.919121+0000","last_undegraded":"2024-09-11T01:15:39.919121+0000","last_fullsized":"2024-09-11T01:15:39.919121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19780067,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1634,"num_write_kb":6536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4802","reported_seq":4926,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.918922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:39.918922+0000","last_peered":"2024-09-11T01:15:39.918922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:39.918922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648995,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1290,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5025","reported_seq":5148,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.956162+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:39.956162+0000","last_peered":"2024-09-11T01:15:39.956162+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:39.956162+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20558790,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1778,"num_write_kb":7112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4845","reported_seq":4954,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.954425+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:39.954425+0000","last_peered":"2024-09-11T01:15:39.954425+0000","last_clean":"2024-09-11T01:15:39.954425+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:39.954425+0000","last_undegraded":"2024-09-11T01:15:39.954425+0000","last_fullsized":"2024-09-11T01:15:39.954425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19836928,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1798,"num_write_kb":7192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4814","reported_seq":5836,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.712895+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:39.712895+0000","last_peered":"2024-09-11T01:15:39.712895+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:39.712895+0000","last_undegraded":"2024-09-11T01:15:39.712895+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4814,"log_dups_size":0,"ondisk_log_size":4814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689955,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":1945600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3649},{"shards":"2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4855","reported_seq":4974,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.921147+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:39.921147+0000","last_peered":"2024-09-11T01:15:39.921147+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:39.921147+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1219,"num_write_kb":4876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":742},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4751","reported_seq":4835,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.690037+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:39.690037+0000","last_peered":"2024-09-11T01:15:39.690037+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:39.690037+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440099,"num_objects":4747,"num_object_clones":0,"num_object_copies":9494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4898","reported_seq":4981,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.662739+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:39.662739+0000","last_peered":"2024-09-11T01:15:39.662739+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:39.662739+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20037632,"num_objects":4892,"num_object_clones":0,"num_object_copies":9784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4887","reported_seq":5007,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.997651+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:39.997651+0000","last_peered":"2024-09-11T01:15:39.997651+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:39.997651+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":721},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4953","reported_seq":5077,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.993093+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:39.993093+0000","last_peered":"2024-09-11T01:15:39.993093+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:39.993093+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4915","reported_seq":5038,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.982026+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:39.982026+0000","last_peered":"2024-09-11T01:15:39.982026+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:39.982026+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4915,"log_dups_size":0,"ondisk_log_size":4915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":763},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4838","reported_seq":4963,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.952973+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:39.952973+0000","last_peered":"2024-09-11T01:15:39.952973+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:39.952973+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19768262,"num_objects":4828,"num_object_clones":0,"num_object_copies":9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1495,"num_write_kb":5980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":674},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317641438,"num_objects":77558,"num_object_clones":0,"num_object_copies":155116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77558,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36693,"num_write_kb":146772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":1949696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":641626112,"data_stored":641470753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77630,"ondisk_log_size":77630,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439172,"kb_used_data":286708,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93932668,"statfs":{"total":96636764160,"available":96187052032,"internally_reserved":0,"allocated":293588992,"data_stored":293332088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789958,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152600,"kb_used_data":40392,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94219240,"statfs":{"total":96636764160,"available":96480501760,"internally_reserved":0,"allocated":41361408,"data_stored":41053458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,1,7,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456912,"kb_used_data":298944,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93914928,"statfs":{"total":96636764160,"available":96168886272,"internally_reserved":0,"allocated":306118656,"data_stored":305857675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306241536,"data_stored":306205406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41451520,"data_stored":41368421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293933056,"data_stored":293896926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:43.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 67 19 79459 79440 4.6311 2.23438 0.015742 0.0168609 2024-09-11T01:15:44.318 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 68 20 80074 80054 4.59825 2.39844 0.0288372 0.0169802 2024-09-11T01:15:45.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 69 20 80683 80663 4.56592 2.37891 0.00884775 0.017102 2024-09-11T01:15:45.881 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:46.321 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 70 20 81253 81233 4.5325 2.22656 0.0540348 0.0172279 2024-09-11T01:15:46.431 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:46.432 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:46.459 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-11T01:15:44.426641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317641457,"num_objects":77560,"num_object_clones":0,"num_object_copies":155120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77560,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36695,"num_write_kb":146774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":1949715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77633,"ondisk_log_size":77633,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1048684,"kb_used_data":626044,"kb_used_omap":12,"kb_used_meta":422579,"kb_avail":282066836,"statfs":{"total":289910292480,"available":288836440064,"internally_reserved":0,"allocated":641069056,"data_stored":640243221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":432720903},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,1,5,11,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5921,"num_write_kb":23684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002898"},"pg_stats":[{"pgid":"2.f","version":"39'4839","reported_seq":4902,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.974972+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:39.974972+0000","last_peered":"2024-09-11T01:15:39.974972+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:39.974972+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4839,"log_dups_size":0,"ondisk_log_size":4839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19820544,"num_objects":4839,"num_object_clones":0,"num_object_copies":9678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4839,"num_write_kb":19356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":716},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'4847","reported_seq":4917,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.602011+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:39.602011+0000","last_peered":"2024-09-11T01:15:39.602011+0000","last_clean":"2024-09-11T01:15:39.602011+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:39.602011+0000","last_undegraded":"2024-09-11T01:15:39.602011+0000","last_fullsized":"2024-09-11T01:15:39.602011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":4847,"log_dups_size":0,"ondisk_log_size":4847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19829702,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2945,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4714","reported_seq":4786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.698124+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:39.698124+0000","last_peered":"2024-09-11T01:15:39.698124+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:39.698124+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4714,"log_dups_size":0,"ondisk_log_size":4714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292160,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2775,"num_write_kb":11100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4811","reported_seq":4846,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.701982+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:39.701982+0000","last_peered":"2024-09-11T01:15:39.701982+0000","last_clean":"2024-09-11T01:15:39.701982+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:39.701982+0000","last_undegraded":"2024-09-11T01:15:39.701982+0000","last_fullsized":"2024-09-11T01:15:39.701982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19705856,"num_objects":4811,"num_object_clones":0,"num_object_copies":9622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4811,"num_write_kb":19244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'4836","reported_seq":4947,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.919121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:39.919121+0000","last_peered":"2024-09-11T01:15:39.919121+0000","last_clean":"2024-09-11T01:15:39.919121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:39.919121+0000","last_undegraded":"2024-09-11T01:15:39.919121+0000","last_fullsized":"2024-09-11T01:15:39.919121+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19780067,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1634,"num_write_kb":6536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4802","reported_seq":4926,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.918922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:39.918922+0000","last_peered":"2024-09-11T01:15:39.918922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:39.918922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4802,"log_dups_size":0,"ondisk_log_size":4802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19648995,"num_objects":4798,"num_object_clones":0,"num_object_copies":9596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4798,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1290,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":683},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5025","reported_seq":5148,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.956162+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:39.956162+0000","last_peered":"2024-09-11T01:15:39.956162+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:39.956162+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5025,"log_dups_size":0,"ondisk_log_size":5025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20558790,"num_objects":5021,"num_object_clones":0,"num_object_copies":10042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5021,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1778,"num_write_kb":7112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'4845","reported_seq":4954,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.954425+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:39.954425+0000","last_peered":"2024-09-11T01:15:39.954425+0000","last_clean":"2024-09-11T01:15:39.954425+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:39.954425+0000","last_undegraded":"2024-09-11T01:15:39.954425+0000","last_fullsized":"2024-09-11T01:15:39.954425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19836928,"num_objects":4843,"num_object_clones":0,"num_object_copies":9686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1798,"num_write_kb":7192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4814","reported_seq":5836,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.712895+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:39.712895+0000","last_peered":"2024-09-11T01:15:39.712895+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:39.712895+0000","last_undegraded":"2024-09-11T01:15:39.712895+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4814,"log_dups_size":0,"ondisk_log_size":4814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689955,"num_objects":4808,"num_object_clones":0,"num_object_copies":9616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":4808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2681,"num_write_kb":10724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":1945600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3649},{"shards":"2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'4855","reported_seq":4974,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.921147+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:39.921147+0000","last_peered":"2024-09-11T01:15:39.921147+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:39.921147+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":4855,"log_dups_size":0,"ondisk_log_size":4855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19853312,"num_objects":4847,"num_object_clones":0,"num_object_copies":9694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1219,"num_write_kb":4876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":742},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4751","reported_seq":4835,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.690037+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:39.690037+0000","last_peered":"2024-09-11T01:15:39.690037+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:39.690037+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4751,"log_dups_size":0,"ondisk_log_size":4751,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19440099,"num_objects":4747,"num_object_clones":0,"num_object_copies":9494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4747,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2520,"num_write_kb":10080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":720}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'4898","reported_seq":4981,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.662739+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:39.662739+0000","last_peered":"2024-09-11T01:15:39.662739+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:39.662739+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":4898,"log_dups_size":0,"ondisk_log_size":4898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20037632,"num_objects":4892,"num_object_clones":0,"num_object_copies":9784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":715}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'4887","reported_seq":5007,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.997651+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:39.997651+0000","last_peered":"2024-09-11T01:15:39.997651+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:39.997651+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":4887,"log_dups_size":0,"ondisk_log_size":4887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20008960,"num_objects":4885,"num_object_clones":0,"num_object_copies":9770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1577,"num_write_kb":6308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":721},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'4953","reported_seq":5077,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.993093+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:39.993093+0000","last_peered":"2024-09-11T01:15:39.993093+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:39.993093+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20254720,"num_objects":4945,"num_object_clones":0,"num_object_copies":9890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1346,"num_write_kb":5384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":753},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'4915","reported_seq":5038,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.982026+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:39.982026+0000","last_peered":"2024-09-11T01:15:39.982026+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:39.982026+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":4915,"log_dups_size":0,"ondisk_log_size":4915,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115456,"num_objects":4911,"num_object_clones":0,"num_object_copies":9822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1450,"num_write_kb":5800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":763},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'4838","reported_seq":4963,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.952973+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:39.952973+0000","last_peered":"2024-09-11T01:15:39.952973+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:39.952973+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":4838,"log_dups_size":0,"ondisk_log_size":4838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19768262,"num_objects":4828,"num_object_clones":0,"num_object_copies":9656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1495,"num_write_kb":5980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":674},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317641438,"num_objects":77558,"num_object_clones":0,"num_object_copies":155116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":77558,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":36693,"num_write_kb":146772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":476,"num_bytes_recovered":1949696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":641626112,"data_stored":641470753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77630,"ondisk_log_size":77630,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439172,"kb_used_data":286708,"kb_used_omap":4,"kb_used_meta":152443,"kb_avail":93932668,"statfs":{"total":96636764160,"available":96187052032,"internally_reserved":0,"allocated":293588992,"data_stored":293332088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":156102318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789958,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152600,"kb_used_data":40392,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94219240,"statfs":{"total":96636764160,"available":96480501760,"internally_reserved":0,"allocated":41361408,"data_stored":41053458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,1,7,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456912,"kb_used_data":298944,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93914928,"statfs":{"total":96636764160,"available":96168886272,"internally_reserved":0,"allocated":306118656,"data_stored":305857675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306241536,"data_stored":306205406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41451520,"data_stored":41368421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293933056,"data_stored":293896926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:46.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:47.038 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:47.039 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:47.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-11T01:15:46.428591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329360113,"num_objects":80421,"num_object_clones":0,"num_object_copies":160842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3526,"num_objects_unfound":0,"num_objects_dirty":80421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":39556,"num_write_kb":158218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":2453523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80494,"ondisk_log_size":80494,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1054784,"kb_used_data":640464,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282060736,"statfs":{"total":289910292480,"available":288830193664,"internally_reserved":0,"allocated":655835136,"data_stored":655009301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":424201223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13,2,5,3,9,15,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35971072,"num_objects":8782,"num_object_clones":0,"num_object_copies":17564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-376,"num_objects_unfound":0,"num_objects_dirty":8782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8782,"num_write_kb":35128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"2.f","version":"39'5024","reported_seq":5087,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.922934+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:44.922934+0000","last_peered":"2024-09-11T01:15:44.922934+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:44.922934+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5024,"log_dups_size":0,"ondisk_log_size":5024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5024,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":901},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5030","reported_seq":5100,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.658056+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:44.658056+0000","last_peered":"2024-09-11T01:15:44.658056+0000","last_clean":"2024-09-11T01:15:44.658056+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:44.658056+0000","last_undegraded":"2024-09-11T01:15:44.658056+0000","last_fullsized":"2024-09-11T01:15:44.658056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20579270,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3128,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4910","reported_seq":4983,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.716320+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:44.716320+0000","last_peered":"2024-09-11T01:15:44.716320+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:44.716320+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20094976,"num_objects":4906,"num_object_clones":0,"num_object_copies":9812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2971,"num_write_kb":11884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":901}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4984","reported_seq":5019,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.646839+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:44.646839+0000","last_peered":"2024-09-11T01:15:44.646839+0000","last_clean":"2024-09-11T01:15:44.646839+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:44.646839+0000","last_undegraded":"2024-09-11T01:15:44.646839+0000","last_fullsized":"2024-09-11T01:15:44.646839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4984,"log_dups_size":0,"ondisk_log_size":4984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20414464,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4984,"num_write_kb":19936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5012","reported_seq":5123,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.961055+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:44.961055+0000","last_peered":"2024-09-11T01:15:44.961055+0000","last_clean":"2024-09-11T01:15:44.961055+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:44.961055+0000","last_undegraded":"2024-09-11T01:15:44.961055+0000","last_fullsized":"2024-09-11T01:15:44.961055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500963,"num_objects":5006,"num_object_clones":0,"num_object_copies":10012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4994","reported_seq":5117,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.992207+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:44.992207+0000","last_peered":"2024-09-11T01:15:44.992207+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:44.992207+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4994,"log_dups_size":0,"ondisk_log_size":4994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20435427,"num_objects":4990,"num_object_clones":0,"num_object_copies":9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1482,"num_write_kb":5928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":875},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5206","reported_seq":5329,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.954583+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:44.954583+0000","last_peered":"2024-09-11T01:15:44.954583+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:44.954583+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5206,"log_dups_size":0,"ondisk_log_size":5206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21300166,"num_objects":5202,"num_object_clones":0,"num_object_copies":10404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1959,"num_write_kb":7836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":919},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5009","reported_seq":5118,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.914913+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:44.914913+0000","last_peered":"2024-09-11T01:15:44.914913+0000","last_clean":"2024-09-11T01:15:44.914913+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:44.914913+0000","last_undegraded":"2024-09-11T01:15:44.914913+0000","last_fullsized":"2024-09-11T01:15:44.914913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1962,"num_write_kb":7848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4990","reported_seq":6259,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:44.711432+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:44.711432+0000","last_peered":"2024-09-11T01:15:44.711432+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:44.711432+0000","last_undegraded":"2024-09-11T01:15:44.711432+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20410851,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3526,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2857,"num_write_kb":11428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":2449408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3526},{"shards":"2","objects":1458}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5053","reported_seq":5172,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:45.003099+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:45.003099+0000","last_peered":"2024-09-11T01:15:45.003099+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:45.003099+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5053,"log_dups_size":0,"ondisk_log_size":5053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664320,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1417,"num_write_kb":5668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":940},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4913","reported_seq":4997,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.722339+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:44.722339+0000","last_peered":"2024-09-11T01:15:44.722339+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:44.722339+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20103651,"num_objects":4909,"num_object_clones":0,"num_object_copies":9818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2682,"num_write_kb":10728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":882}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5077","reported_seq":5160,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.685815+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:44.685815+0000","last_peered":"2024-09-11T01:15:44.685815+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:44.685815+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770816,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":894}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5063","reported_seq":5183,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.952125+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:44.952125+0000","last_peered":"2024-09-11T01:15:44.952125+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:44.952125+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5063,"log_dups_size":0,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20729856,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1753,"num_write_kb":7012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":897},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5130","reported_seq":5254,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.905597+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:44.905597+0000","last_peered":"2024-09-11T01:15:44.905597+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:44.905597+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5130,"log_dups_size":0,"ondisk_log_size":5130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1523,"num_write_kb":6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5085","reported_seq":5208,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.982427+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:44.982427+0000","last_peered":"2024-09-11T01:15:44.982427+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:44.982427+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5085,"log_dups_size":0,"ondisk_log_size":5085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1620,"num_write_kb":6480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5011","reported_seq":5136,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.987041+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:44.987041+0000","last_peered":"2024-09-11T01:15:44.987041+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:44.987041+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20476870,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1668,"num_write_kb":6672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":847},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329360094,"num_objects":80419,"num_object_clones":0,"num_object_copies":160838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3526,"num_objects_unfound":0,"num_objects_dirty":80419,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":39554,"num_write_kb":158216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":2453504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":656134144,"data_stored":655978785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80491,"ondisk_log_size":80491,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432440,"kb_used_data":292456,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93939400,"statfs":{"total":96636764160,"available":96193945600,"internally_reserved":0,"allocated":299474944,"data_stored":299218040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1,0,2,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789958,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152600,"kb_used_data":40392,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94219240,"statfs":{"total":96636764160,"available":96480501760,"internally_reserved":0,"allocated":41361408,"data_stored":41053458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,1,7,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469744,"kb_used_data":307616,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93902096,"statfs":{"total":96636764160,"available":96155746304,"internally_reserved":0,"allocated":314998784,"data_stored":314737803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,9,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315006976,"data_stored":314970846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41451520,"data_stored":41368421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299675648,"data_stored":299639518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:47.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'5024", 'reported_seq': 5087, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.922934+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:15:44.922934+0000', 'last_peered': '2024-09-11T01:15:44.922934+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:15:44.922934+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 5024, 'log_dups_size': 0, 'ondisk_log_size': 5024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20578304, 'num_objects': 5024, 'num_object_clones': 0, 'num_object_copies': 10048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5024, 'num_write_kb': 20096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 901}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'4910", 'reported_seq': 4983, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.716320+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:44.716320+0000', 'last_peered': '2024-09-11T01:15:44.716320+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:44.716320+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 4910, 'log_dups_size': 0, 'ondisk_log_size': 4910, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20094976, 'num_objects': 4906, 'num_object_clones': 0, 'num_object_copies': 9812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2971, 'num_write_kb': 11884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 901}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'4994", 'reported_seq': 5117, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.992207+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:44.992207+0000', 'last_peered': '2024-09-11T01:15:44.992207+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:44.992207+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 4994, 'log_dups_size': 0, 'ondisk_log_size': 4994, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20435427, 'num_objects': 4990, 'num_object_clones': 0, 'num_object_copies': 9980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4990, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1482, 'num_write_kb': 5928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 875}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'5206", 'reported_seq': 5329, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.954583+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:15:44.954583+0000', 'last_peered': '2024-09-11T01:15:44.954583+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:15:44.954583+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 5206, 'log_dups_size': 0, 'ondisk_log_size': 5206, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21300166, 'num_objects': 5202, 'num_object_clones': 0, 'num_object_copies': 10404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5202, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 1959, 'num_write_kb': 7836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 919}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:47.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'4990", 'reported_seq': 6259, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:44.711432+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:44.711432+0000', 'last_peered': '2024-09-11T01:15:44.711432+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:44.711432+0000', 'last_undegraded': '2024-09-11T01:15:44.711432+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 4990, 'log_dups_size': 0, 'ondisk_log_size': 4990, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20410851, 'num_objects': 4984, 'num_object_clones': 0, 'num_object_copies': 9968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3526, 'num_objects_unfound': 0, 'num_objects_dirty': 4984, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2857, 'num_write_kb': 11428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 598, 'num_bytes_recovered': 2449408, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3526}, {'shards': '2', 'objects': 1458}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'5053", 'reported_seq': 5172, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:45.003099+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:15:45.003099+0000', 'last_peered': '2024-09-11T01:15:45.003099+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:15:45.003099+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 5053, 'log_dups_size': 0, 'ondisk_log_size': 5053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20664320, 'num_objects': 5045, 'num_object_clones': 0, 'num_object_copies': 10090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1417, 'num_write_kb': 5668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 940}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'4913", 'reported_seq': 4997, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.722339+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:44.722339+0000', 'last_peered': '2024-09-11T01:15:44.722339+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:44.722339+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 4913, 'log_dups_size': 0, 'ondisk_log_size': 4913, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20103651, 'num_objects': 4909, 'num_object_clones': 0, 'num_object_copies': 9818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4909, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2682, 'num_write_kb': 10728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 882}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'5077", 'reported_seq': 5160, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.685815+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:44.685815+0000', 'last_peered': '2024-09-11T01:15:44.685815+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:44.685815+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 5077, 'log_dups_size': 0, 'ondisk_log_size': 5077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20770816, 'num_objects': 5071, 'num_object_clones': 0, 'num_object_copies': 10142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2714, 'num_write_kb': 10856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 894}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'5063", 'reported_seq': 5183, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.952125+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:15:44.952125+0000', 'last_peered': '2024-09-11T01:15:44.952125+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:15:44.952125+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 5063, 'log_dups_size': 0, 'ondisk_log_size': 5063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20729856, 'num_objects': 5061, 'num_object_clones': 0, 'num_object_copies': 10122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1753, 'num_write_kb': 7012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 897}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'5130", 'reported_seq': 5254, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.905597+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:15:44.905597+0000', 'last_peered': '2024-09-11T01:15:44.905597+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:15:44.905597+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 5130, 'log_dups_size': 0, 'ondisk_log_size': 5130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20979712, 'num_objects': 5122, 'num_object_clones': 0, 'num_object_copies': 10244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1523, 'num_write_kb': 6092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 930}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:47.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'5085", 'reported_seq': 5208, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.982427+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:44.982427+0000', 'last_peered': '2024-09-11T01:15:44.982427+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:44.982427+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 5085, 'log_dups_size': 0, 'ondisk_log_size': 5085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20811776, 'num_objects': 5081, 'num_object_clones': 0, 'num_object_copies': 10162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1620, 'num_write_kb': 6480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 933}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:47.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'5011", 'reported_seq': 5136, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:44.987041+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:15:44.987041+0000', 'last_peered': '2024-09-11T01:15:44.987041+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:15:44.987041+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 5011, 'log_dups_size': 0, 'ondisk_log_size': 5011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20476870, 'num_objects': 5001, 'num_object_clones': 0, 'num_object_copies': 10002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5001, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1668, 'num_write_kb': 6672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 847}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:47.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:47.322 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 71 20 81787 81767 4.49797 2.08594 0.0794746 0.0173605 2024-09-11T01:15:47.761 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:47.761 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/160842 objects degraded (28.294%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":148,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80421,"data_bytes":329360113,"bytes_used":1080098816,"bytes_avail":288830193664,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":160842,"degraded_ratio":0.28293604904191694,"misplaced_objects":3526,"misplaced_total":160842,"misplaced_ratio":0.021922134765795007,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":128294,"recovering_keys_per_sec":0,"num_objects_recovered":376,"num_bytes_recovered":1540096,"num_keys_recovered":0,"write_bytes_sec":2996497,"read_op_per_sec":0,"write_op_per_sec":731},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:47.792 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:47.792 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:48.324 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 72 20 82300 82280 4.4632 2.00391 0.0151932 0.0174941 2024-09-11T01:15:48.344 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:48.345 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:48.371 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-11T01:15:46.428591+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329360113,"num_objects":80421,"num_object_clones":0,"num_object_copies":160842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3526,"num_objects_unfound":0,"num_objects_dirty":80421,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":39556,"num_write_kb":158218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":2453523,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80494,"ondisk_log_size":80494,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1054784,"kb_used_data":640464,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282060736,"statfs":{"total":289910292480,"available":288830193664,"internally_reserved":0,"allocated":655835136,"data_stored":655009301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":424201223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[13,2,5,3,9,15,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35971072,"num_objects":8782,"num_object_clones":0,"num_object_copies":17564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-376,"num_objects_unfound":0,"num_objects_dirty":8782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8782,"num_write_kb":35128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":376,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004372"},"pg_stats":[{"pgid":"2.f","version":"39'5024","reported_seq":5087,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.922934+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:44.922934+0000","last_peered":"2024-09-11T01:15:44.922934+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:44.922934+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5024,"log_dups_size":0,"ondisk_log_size":5024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20578304,"num_objects":5024,"num_object_clones":0,"num_object_copies":10048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5024,"num_write_kb":20096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":901},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5030","reported_seq":5100,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.658056+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:44.658056+0000","last_peered":"2024-09-11T01:15:44.658056+0000","last_clean":"2024-09-11T01:15:44.658056+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:44.658056+0000","last_undegraded":"2024-09-11T01:15:44.658056+0000","last_fullsized":"2024-09-11T01:15:44.658056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5030,"log_dups_size":0,"ondisk_log_size":5030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20579270,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3128,"num_write_kb":12512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'4910","reported_seq":4983,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.716320+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:44.716320+0000","last_peered":"2024-09-11T01:15:44.716320+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:44.716320+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":4910,"log_dups_size":0,"ondisk_log_size":4910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20094976,"num_objects":4906,"num_object_clones":0,"num_object_copies":9812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2971,"num_write_kb":11884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":901}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'4984","reported_seq":5019,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.646839+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:44.646839+0000","last_peered":"2024-09-11T01:15:44.646839+0000","last_clean":"2024-09-11T01:15:44.646839+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:44.646839+0000","last_undegraded":"2024-09-11T01:15:44.646839+0000","last_fullsized":"2024-09-11T01:15:44.646839+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":4984,"log_dups_size":0,"ondisk_log_size":4984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20414464,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4984,"num_write_kb":19936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5012","reported_seq":5123,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.961055+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:44.961055+0000","last_peered":"2024-09-11T01:15:44.961055+0000","last_clean":"2024-09-11T01:15:44.961055+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:44.961055+0000","last_undegraded":"2024-09-11T01:15:44.961055+0000","last_fullsized":"2024-09-11T01:15:44.961055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500963,"num_objects":5006,"num_object_clones":0,"num_object_copies":10012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'4994","reported_seq":5117,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.992207+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:44.992207+0000","last_peered":"2024-09-11T01:15:44.992207+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:44.992207+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":4994,"log_dups_size":0,"ondisk_log_size":4994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20435427,"num_objects":4990,"num_object_clones":0,"num_object_copies":9980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1482,"num_write_kb":5928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":875},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5206","reported_seq":5329,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.954583+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:44.954583+0000","last_peered":"2024-09-11T01:15:44.954583+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:44.954583+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5206,"log_dups_size":0,"ondisk_log_size":5206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21300166,"num_objects":5202,"num_object_clones":0,"num_object_copies":10404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5202,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1959,"num_write_kb":7836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":919},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5009","reported_seq":5118,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.914913+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:44.914913+0000","last_peered":"2024-09-11T01:15:44.914913+0000","last_clean":"2024-09-11T01:15:44.914913+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:44.914913+0000","last_undegraded":"2024-09-11T01:15:44.914913+0000","last_fullsized":"2024-09-11T01:15:44.914913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5009,"log_dups_size":0,"ondisk_log_size":5009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1962,"num_write_kb":7848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'4990","reported_seq":6259,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:44.711432+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:44.711432+0000","last_peered":"2024-09-11T01:15:44.711432+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:44.711432+0000","last_undegraded":"2024-09-11T01:15:44.711432+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20410851,"num_objects":4984,"num_object_clones":0,"num_object_copies":9968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3526,"num_objects_unfound":0,"num_objects_dirty":4984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2857,"num_write_kb":11428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":598,"num_bytes_recovered":2449408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3526},{"shards":"2","objects":1458}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5053","reported_seq":5172,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:45.003099+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:45.003099+0000","last_peered":"2024-09-11T01:15:45.003099+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:45.003099+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5053,"log_dups_size":0,"ondisk_log_size":5053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664320,"num_objects":5045,"num_object_clones":0,"num_object_copies":10090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1417,"num_write_kb":5668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":940},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'4913","reported_seq":4997,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.722339+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:44.722339+0000","last_peered":"2024-09-11T01:15:44.722339+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:44.722339+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20103651,"num_objects":4909,"num_object_clones":0,"num_object_copies":9818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4909,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2682,"num_write_kb":10728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":882}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5077","reported_seq":5160,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.685815+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:44.685815+0000","last_peered":"2024-09-11T01:15:44.685815+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:44.685815+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5077,"log_dups_size":0,"ondisk_log_size":5077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20770816,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2714,"num_write_kb":10856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":894}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5063","reported_seq":5183,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.952125+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:44.952125+0000","last_peered":"2024-09-11T01:15:44.952125+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:44.952125+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5063,"log_dups_size":0,"ondisk_log_size":5063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20729856,"num_objects":5061,"num_object_clones":0,"num_object_copies":10122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1753,"num_write_kb":7012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":897},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5130","reported_seq":5254,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.905597+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:44.905597+0000","last_peered":"2024-09-11T01:15:44.905597+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:44.905597+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5130,"log_dups_size":0,"ondisk_log_size":5130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1523,"num_write_kb":6092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":930},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5085","reported_seq":5208,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.982427+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:44.982427+0000","last_peered":"2024-09-11T01:15:44.982427+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:44.982427+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5085,"log_dups_size":0,"ondisk_log_size":5085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1620,"num_write_kb":6480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5011","reported_seq":5136,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.987041+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:44.987041+0000","last_peered":"2024-09-11T01:15:44.987041+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:44.987041+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5011,"log_dups_size":0,"ondisk_log_size":5011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20476870,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1668,"num_write_kb":6672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":847},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329360094,"num_objects":80419,"num_object_clones":0,"num_object_copies":160838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3526,"num_objects_unfound":0,"num_objects_dirty":80419,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":39554,"num_write_kb":158216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":599,"num_bytes_recovered":2453504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":656134144,"data_stored":655978785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80491,"ondisk_log_size":80491,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432440,"kb_used_data":292456,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93939400,"statfs":{"total":96636764160,"available":96193945600,"internally_reserved":0,"allocated":299474944,"data_stored":299218040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":143322798},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1,0,2,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789958,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152600,"kb_used_data":40392,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94219240,"statfs":{"total":96636764160,"available":96480501760,"internally_reserved":0,"allocated":41361408,"data_stored":41053458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,1,7,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469744,"kb_used_data":307616,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93902096,"statfs":{"total":96636764160,"available":96155746304,"internally_reserved":0,"allocated":314998784,"data_stored":314737803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,9,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315006976,"data_stored":314970846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41451520,"data_stored":41368421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":299675648,"data_stored":299639518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:49.324 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 73 20 82898 82878 4.43406 2.33594 0.014634 0.0176114 2024-09-11T01:15:50.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 74 20 83494 83474 4.40555 2.32812 0.0318976 0.0177247 2024-09-11T01:15:51.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 75 20 84084 84064 4.37754 2.30469 0.0310436 0.0178385 2024-09-11T01:15:51.374 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:51.979 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:51.981 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:52.004 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-11T01:15:50.429450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":341025521,"num_objects":83269,"num_object_clones":0,"num_object_copies":166538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":83269,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":42404,"num_write_kb":169610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":718,"num_bytes_recovered":2936851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83342,"ondisk_log_size":83342,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1092252,"kb_used_data":664044,"kb_used_omap":12,"kb_used_meta":428147,"kb_avail":282023268,"statfs":{"total":289910292480,"available":288791826432,"internally_reserved":0,"allocated":679981056,"data_stored":679155221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":438422535},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,5,5,9,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35495936,"num_objects":8666,"num_object_clones":0,"num_object_copies":17332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-365,"num_objects_unfound":0,"num_objects_dirty":8666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8666,"num_write_kb":34664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004311"},"pg_stats":[{"pgid":"2.f","version":"39'5176","reported_seq":5236,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.969754+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:49.969754+0000","last_peered":"2024-09-11T01:15:49.969754+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:49.969754+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5217","reported_seq":5287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.727344+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:49.727344+0000","last_peered":"2024-09-11T01:15:49.727344+0000","last_clean":"2024-09-11T01:15:49.727344+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:49.727344+0000","last_undegraded":"2024-09-11T01:15:49.727344+0000","last_fullsized":"2024-09-11T01:15:49.727344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21345222,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3315,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5085","reported_seq":5158,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.715953+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:49.715953+0000","last_peered":"2024-09-11T01:15:49.715953+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:49.715953+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5085,"log_dups_size":0,"ondisk_log_size":5085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3146,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5160","reported_seq":5195,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.715952+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:49.715952+0000","last_peered":"2024-09-11T01:15:49.715952+0000","last_clean":"2024-09-11T01:15:49.715952+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:49.715952+0000","last_undegraded":"2024-09-11T01:15:49.715952+0000","last_fullsized":"2024-09-11T01:15:49.715952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5160,"log_dups_size":0,"ondisk_log_size":5160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5160,"num_write_kb":20640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5191","reported_seq":5302,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.937040+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:49.937040+0000","last_peered":"2024-09-11T01:15:49.937040+0000","last_clean":"2024-09-11T01:15:49.937040+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:49.937040+0000","last_undegraded":"2024-09-11T01:15:49.937040+0000","last_fullsized":"2024-09-11T01:15:49.937040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5191,"log_dups_size":0,"ondisk_log_size":5191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21234147,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1989,"num_write_kb":7956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5174","reported_seq":5298,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.985245+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:49.985245+0000","last_peered":"2024-09-11T01:15:49.985245+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:49.985245+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172707,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1662,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5394","reported_seq":5513,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.020399+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:50.020399+0000","last_peered":"2024-09-11T01:15:50.020399+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:50.020399+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22070214,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1107},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5164","reported_seq":5273,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.980848+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:49.980848+0000","last_peered":"2024-09-11T01:15:49.980848+0000","last_clean":"2024-09-11T01:15:49.980848+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:49.980848+0000","last_undegraded":"2024-09-11T01:15:49.980848+0000","last_fullsized":"2024-09-11T01:15:49.980848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5164,"log_dups_size":0,"ondisk_log_size":5164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143552,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2117,"num_write_kb":8468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5171","reported_seq":6676,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:49.741182+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:49.741182+0000","last_peered":"2024-09-11T01:15:49.741182+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:49.741182+0000","last_undegraded":"2024-09-11T01:15:49.741182+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5171,"log_dups_size":0,"ondisk_log_size":5171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":716,"num_bytes_recovered":2932736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3408},{"shards":"2","objects":1757}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5249","reported_seq":5368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.019204+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:50.019204+0000","last_peered":"2024-09-11T01:15:50.019204+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:50.019204+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5249,"log_dups_size":0,"ondisk_log_size":5249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21467136,"num_objects":5241,"num_object_clones":0,"num_object_copies":10482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1613,"num_write_kb":6452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1136},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5094","reported_seq":5178,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.705931+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:49.705931+0000","last_peered":"2024-09-11T01:15:49.705931+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:49.705931+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20845027,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1063}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5242","reported_seq":5325,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.721159+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:49.721159+0000","last_peered":"2024-09-11T01:15:49.721159+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:49.721159+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2879,"num_write_kb":11516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1059}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5239","reported_seq":5358,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.973038+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:49.973038+0000","last_peered":"2024-09-11T01:15:49.973038+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:49.973038+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1929,"num_write_kb":7716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5327","reported_seq":5451,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.009802+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:50.009802+0000","last_peered":"2024-09-11T01:15:50.009802+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:50.009802+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1720,"num_write_kb":6880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1127},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5275","reported_seq":5398,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.020425+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:50.020425+0000","last_peered":"2024-09-11T01:15:50.020425+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:50.020425+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5275,"log_dups_size":0,"ondisk_log_size":5275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21590016,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5181","reported_seq":5306,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.932748+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:49.932748+0000","last_peered":"2024-09-11T01:15:49.932748+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:49.932748+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5181,"log_dups_size":0,"ondisk_log_size":5181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173190,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":341025502,"num_objects":83267,"num_object_clones":0,"num_object_copies":166534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":83267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":42402,"num_write_kb":169608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":2936832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":680128512,"data_stored":679973153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83339,"ondisk_log_size":83339,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460368,"kb_used_data":298176,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93911472,"statfs":{"total":96636764160,"available":96165347328,"internally_reserved":0,"allocated":305332224,"data_stored":305075320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789959,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149436,"kb_used_data":49708,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94222404,"statfs":{"total":96636764160,"available":96483741696,"internally_reserved":0,"allocated":50900992,"data_stored":50593042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":102100655},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482448,"kb_used_data":316160,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93889392,"statfs":{"total":96636764160,"available":96142737408,"internally_reserved":0,"allocated":323747840,"data_stored":323486859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,1,0,1,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":323784704,"data_stored":323748574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50950144,"data_stored":50867045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305393664,"data_stored":305357534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:52.006 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:52.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 76 20 84614 84594 4.34718 2.07031 0.012812 0.0179625 2024-09-11T01:15:52.585 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:52.588 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:52.612 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-11T01:15:52.429895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":341025521,"num_objects":83269,"num_object_clones":0,"num_object_copies":166538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":83269,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":42404,"num_write_kb":169610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":718,"num_bytes_recovered":2936851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83342,"ondisk_log_size":83342,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1109608,"kb_used_data":673080,"kb_used_omap":12,"kb_used_meta":436467,"kb_avail":282005912,"statfs":{"total":289910292480,"available":288774053888,"internally_reserved":0,"allocated":689233920,"data_stored":688408085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":446942215},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,2,2,5,7,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004314"},"pg_stats":[{"pgid":"2.f","version":"39'5176","reported_seq":5236,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.969754+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:49.969754+0000","last_peered":"2024-09-11T01:15:49.969754+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:49.969754+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5217","reported_seq":5287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.727344+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:49.727344+0000","last_peered":"2024-09-11T01:15:49.727344+0000","last_clean":"2024-09-11T01:15:49.727344+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:49.727344+0000","last_undegraded":"2024-09-11T01:15:49.727344+0000","last_fullsized":"2024-09-11T01:15:49.727344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21345222,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3315,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5085","reported_seq":5158,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.715953+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:49.715953+0000","last_peered":"2024-09-11T01:15:49.715953+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:49.715953+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5085,"log_dups_size":0,"ondisk_log_size":5085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3146,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5160","reported_seq":5195,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.715952+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:49.715952+0000","last_peered":"2024-09-11T01:15:49.715952+0000","last_clean":"2024-09-11T01:15:49.715952+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:49.715952+0000","last_undegraded":"2024-09-11T01:15:49.715952+0000","last_fullsized":"2024-09-11T01:15:49.715952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5160,"log_dups_size":0,"ondisk_log_size":5160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5160,"num_write_kb":20640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5191","reported_seq":5302,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.937040+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:49.937040+0000","last_peered":"2024-09-11T01:15:49.937040+0000","last_clean":"2024-09-11T01:15:49.937040+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:49.937040+0000","last_undegraded":"2024-09-11T01:15:49.937040+0000","last_fullsized":"2024-09-11T01:15:49.937040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5191,"log_dups_size":0,"ondisk_log_size":5191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21234147,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1989,"num_write_kb":7956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5174","reported_seq":5298,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.985245+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:49.985245+0000","last_peered":"2024-09-11T01:15:49.985245+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:49.985245+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172707,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1662,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5394","reported_seq":5513,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.020399+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:50.020399+0000","last_peered":"2024-09-11T01:15:50.020399+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:50.020399+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22070214,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1107},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5164","reported_seq":5273,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.980848+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:49.980848+0000","last_peered":"2024-09-11T01:15:49.980848+0000","last_clean":"2024-09-11T01:15:49.980848+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:49.980848+0000","last_undegraded":"2024-09-11T01:15:49.980848+0000","last_fullsized":"2024-09-11T01:15:49.980848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5164,"log_dups_size":0,"ondisk_log_size":5164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143552,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2117,"num_write_kb":8468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5171","reported_seq":6676,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:49.741182+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:49.741182+0000","last_peered":"2024-09-11T01:15:49.741182+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:49.741182+0000","last_undegraded":"2024-09-11T01:15:49.741182+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5171,"log_dups_size":0,"ondisk_log_size":5171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":716,"num_bytes_recovered":2932736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3408},{"shards":"2","objects":1757}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5249","reported_seq":5368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.019204+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:50.019204+0000","last_peered":"2024-09-11T01:15:50.019204+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:50.019204+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5249,"log_dups_size":0,"ondisk_log_size":5249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21467136,"num_objects":5241,"num_object_clones":0,"num_object_copies":10482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1613,"num_write_kb":6452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1136},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5094","reported_seq":5178,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.705931+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:49.705931+0000","last_peered":"2024-09-11T01:15:49.705931+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:49.705931+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20845027,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1063}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5242","reported_seq":5325,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.721159+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:49.721159+0000","last_peered":"2024-09-11T01:15:49.721159+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:49.721159+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2879,"num_write_kb":11516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1059}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5239","reported_seq":5358,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.973038+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:49.973038+0000","last_peered":"2024-09-11T01:15:49.973038+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:49.973038+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1929,"num_write_kb":7716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5327","reported_seq":5451,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.009802+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:50.009802+0000","last_peered":"2024-09-11T01:15:50.009802+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:50.009802+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1720,"num_write_kb":6880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1127},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5275","reported_seq":5398,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.020425+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:50.020425+0000","last_peered":"2024-09-11T01:15:50.020425+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:50.020425+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5275,"log_dups_size":0,"ondisk_log_size":5275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21590016,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5181","reported_seq":5306,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.932748+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:49.932748+0000","last_peered":"2024-09-11T01:15:49.932748+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:49.932748+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5181,"log_dups_size":0,"ondisk_log_size":5181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173190,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":341025502,"num_objects":83267,"num_object_clones":0,"num_object_copies":166534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":83267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":42402,"num_write_kb":169608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":2936832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":689442816,"data_stored":689287457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83339,"ondisk_log_size":83339,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460368,"kb_used_data":298176,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93911472,"statfs":{"total":96636764160,"available":96165347328,"internally_reserved":0,"allocated":305332224,"data_stored":305075320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789960,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166792,"kb_used_data":58744,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94205048,"statfs":{"total":96636764160,"available":96465969152,"internally_reserved":0,"allocated":60153856,"data_stored":59845906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482448,"kb_used_data":316160,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93889392,"statfs":{"total":96636764160,"available":96142737408,"internally_reserved":0,"allocated":323747840,"data_stored":323486859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,1,0,1,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":323784704,"data_stored":323748574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60264448,"data_stored":60181349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305393664,"data_stored":305357534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:52.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:52.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'5176", 'reported_seq': 5236, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.969754+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:15:49.969754+0000', 'last_peered': '2024-09-11T01:15:49.969754+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:15:49.969754+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 5176, 'log_dups_size': 0, 'ondisk_log_size': 5176, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21200896, 'num_objects': 5176, 'num_object_clones': 0, 'num_object_copies': 10352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5176, 'num_write_kb': 20704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1053}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'5085", 'reported_seq': 5158, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.715953+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:49.715953+0000', 'last_peered': '2024-09-11T01:15:49.715953+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:49.715953+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 5085, 'log_dups_size': 0, 'ondisk_log_size': 5085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20811776, 'num_objects': 5081, 'num_object_clones': 0, 'num_object_copies': 10162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5081, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3146, 'num_write_kb': 12584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 1076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'5174", 'reported_seq': 5298, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.985245+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:49.985245+0000', 'last_peered': '2024-09-11T01:15:49.985245+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:49.985245+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 5174, 'log_dups_size': 0, 'ondisk_log_size': 5174, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21172707, 'num_objects': 5170, 'num_object_clones': 0, 'num_object_copies': 10340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5170, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1662, 'num_write_kb': 6648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1055}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'5394", 'reported_seq': 5513, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:50.020399+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:15:50.020399+0000', 'last_peered': '2024-09-11T01:15:50.020399+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:15:50.020399+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 5394, 'log_dups_size': 0, 'ondisk_log_size': 5394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22070214, 'num_objects': 5390, 'num_object_clones': 0, 'num_object_copies': 10780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5390, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2147, 'num_write_kb': 8588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1107}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'5171", 'reported_seq': 6676, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:49.741182+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:49.741182+0000', 'last_peered': '2024-09-11T01:15:49.741182+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:49.741182+0000', 'last_undegraded': '2024-09-11T01:15:49.741182+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 5171, 'log_dups_size': 0, 'ondisk_log_size': 5171, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21152227, 'num_objects': 5165, 'num_object_clones': 0, 'num_object_copies': 10330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3408, 'num_objects_unfound': 0, 'num_objects_dirty': 5165, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3038, 'num_write_kb': 12152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 716, 'num_bytes_recovered': 2932736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3408}, {'shards': '2', 'objects': 1757}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'5249", 'reported_seq': 5368, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:50.019204+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:15:50.019204+0000', 'last_peered': '2024-09-11T01:15:50.019204+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:15:50.019204+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 5249, 'log_dups_size': 0, 'ondisk_log_size': 5249, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21467136, 'num_objects': 5241, 'num_object_clones': 0, 'num_object_copies': 10482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1613, 'num_write_kb': 6452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1136}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:52.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'5094", 'reported_seq': 5178, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.705931+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:49.705931+0000', 'last_peered': '2024-09-11T01:15:49.705931+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:49.705931+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 5094, 'log_dups_size': 0, 'ondisk_log_size': 5094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20845027, 'num_objects': 5090, 'num_object_clones': 0, 'num_object_copies': 10180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5090, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2863, 'num_write_kb': 11452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 1063}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'5242", 'reported_seq': 5325, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.721159+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:49.721159+0000', 'last_peered': '2024-09-11T01:15:49.721159+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:49.721159+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 5242, 'log_dups_size': 0, 'ondisk_log_size': 5242, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21446656, 'num_objects': 5236, 'num_object_clones': 0, 'num_object_copies': 10472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2879, 'num_write_kb': 11516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 1059}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'5239", 'reported_seq': 5358, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.973038+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:15:49.973038+0000', 'last_peered': '2024-09-11T01:15:49.973038+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:15:49.973038+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 5239, 'log_dups_size': 0, 'ondisk_log_size': 5239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21450752, 'num_objects': 5237, 'num_object_clones': 0, 'num_object_copies': 10474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1929, 'num_write_kb': 7716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1073}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'5327", 'reported_seq': 5451, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:50.009802+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:15:50.009802+0000', 'last_peered': '2024-09-11T01:15:50.009802+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:15:50.009802+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 5327, 'log_dups_size': 0, 'ondisk_log_size': 5327, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21786624, 'num_objects': 5319, 'num_object_clones': 0, 'num_object_copies': 10638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1720, 'num_write_kb': 6880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1127}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'5275", 'reported_seq': 5398, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:50.020425+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:50.020425+0000', 'last_peered': '2024-09-11T01:15:50.020425+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:50.020425+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 5275, 'log_dups_size': 0, 'ondisk_log_size': 5275, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21590016, 'num_objects': 5271, 'num_object_clones': 0, 'num_object_copies': 10542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1810, 'num_write_kb': 7240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1123}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:52.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'5181", 'reported_seq': 5306, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.932748+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:15:49.932748+0000', 'last_peered': '2024-09-11T01:15:49.932748+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:15:49.932748+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 5181, 'log_dups_size': 0, 'ondisk_log_size': 5181, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21173190, 'num_objects': 5171, 'num_object_clones': 0, 'num_object_copies': 10342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5171, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1838, 'num_write_kb': 7352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1017}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:52.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:53.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 77 20 85138 85118 4.31731 2.04688 0.0312836 0.0180876 2024-09-11T01:15:53.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:53.357 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/166538 objects degraded (27.326%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":153,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":83269,"data_bytes":341025521,"bytes_used":1136238592,"bytes_avail":288774053888,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":166538,"degraded_ratio":0.2732589559139656,"misplaced_objects":3408,"misplaced_total":166538,"misplaced_ratio":0.020463798052096215,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":82231,"recovering_keys_per_sec":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"write_bytes_sec":1947971,"read_op_per_sec":0,"write_op_per_sec":475},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:53.389 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:53.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:53.985 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:53.987 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:54.016 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-11T01:15:52.429895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":341025521,"num_objects":83269,"num_object_clones":0,"num_object_copies":166538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":83269,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":42404,"num_write_kb":169610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":718,"num_bytes_recovered":2936851,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83342,"ondisk_log_size":83342,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1109608,"kb_used_data":673080,"kb_used_omap":12,"kb_used_meta":436467,"kb_avail":282005912,"statfs":{"total":289910292480,"available":288774053888,"internally_reserved":0,"allocated":689233920,"data_stored":688408085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":446942215},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,2,2,5,7,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23384064,"num_objects":5709,"num_object_clones":0,"num_object_copies":11418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":5709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004314"},"pg_stats":[{"pgid":"2.f","version":"39'5176","reported_seq":5236,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.969754+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:49.969754+0000","last_peered":"2024-09-11T01:15:49.969754+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:49.969754+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5176,"log_dups_size":0,"ondisk_log_size":5176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1053},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5217","reported_seq":5287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.727344+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:49.727344+0000","last_peered":"2024-09-11T01:15:49.727344+0000","last_clean":"2024-09-11T01:15:49.727344+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:49.727344+0000","last_undegraded":"2024-09-11T01:15:49.727344+0000","last_fullsized":"2024-09-11T01:15:49.727344+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21345222,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3315,"num_write_kb":13260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5085","reported_seq":5158,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.715953+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:49.715953+0000","last_peered":"2024-09-11T01:15:49.715953+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:49.715953+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5085,"log_dups_size":0,"ondisk_log_size":5085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3146,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1076}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5160","reported_seq":5195,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.715952+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:49.715952+0000","last_peered":"2024-09-11T01:15:49.715952+0000","last_clean":"2024-09-11T01:15:49.715952+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:49.715952+0000","last_undegraded":"2024-09-11T01:15:49.715952+0000","last_fullsized":"2024-09-11T01:15:49.715952+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5160,"log_dups_size":0,"ondisk_log_size":5160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5160,"num_write_kb":20640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5191","reported_seq":5302,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.937040+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:49.937040+0000","last_peered":"2024-09-11T01:15:49.937040+0000","last_clean":"2024-09-11T01:15:49.937040+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:49.937040+0000","last_undegraded":"2024-09-11T01:15:49.937040+0000","last_fullsized":"2024-09-11T01:15:49.937040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5191,"log_dups_size":0,"ondisk_log_size":5191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21234147,"num_objects":5185,"num_object_clones":0,"num_object_copies":10370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1989,"num_write_kb":7956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5174","reported_seq":5298,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.985245+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:49.985245+0000","last_peered":"2024-09-11T01:15:49.985245+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:49.985245+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5174,"log_dups_size":0,"ondisk_log_size":5174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21172707,"num_objects":5170,"num_object_clones":0,"num_object_copies":10340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1662,"num_write_kb":6648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1055},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5394","reported_seq":5513,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.020399+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:50.020399+0000","last_peered":"2024-09-11T01:15:50.020399+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:50.020399+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22070214,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2147,"num_write_kb":8588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1107},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5164","reported_seq":5273,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.980848+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:49.980848+0000","last_peered":"2024-09-11T01:15:49.980848+0000","last_clean":"2024-09-11T01:15:49.980848+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:49.980848+0000","last_undegraded":"2024-09-11T01:15:49.980848+0000","last_fullsized":"2024-09-11T01:15:49.980848+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5164,"log_dups_size":0,"ondisk_log_size":5164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21143552,"num_objects":5162,"num_object_clones":0,"num_object_copies":10324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2117,"num_write_kb":8468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5171","reported_seq":6676,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:49.741182+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:49.741182+0000","last_peered":"2024-09-11T01:15:49.741182+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:49.741182+0000","last_undegraded":"2024-09-11T01:15:49.741182+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5171,"log_dups_size":0,"ondisk_log_size":5171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21152227,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":716,"num_bytes_recovered":2932736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3408},{"shards":"2","objects":1757}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5249","reported_seq":5368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.019204+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:50.019204+0000","last_peered":"2024-09-11T01:15:50.019204+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:50.019204+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5249,"log_dups_size":0,"ondisk_log_size":5249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21467136,"num_objects":5241,"num_object_clones":0,"num_object_copies":10482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1613,"num_write_kb":6452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1136},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5094","reported_seq":5178,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.705931+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:49.705931+0000","last_peered":"2024-09-11T01:15:49.705931+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:49.705931+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5094,"log_dups_size":0,"ondisk_log_size":5094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20845027,"num_objects":5090,"num_object_clones":0,"num_object_copies":10180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2863,"num_write_kb":11452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1063}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5242","reported_seq":5325,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.721159+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:49.721159+0000","last_peered":"2024-09-11T01:15:49.721159+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:49.721159+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5242,"log_dups_size":0,"ondisk_log_size":5242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2879,"num_write_kb":11516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1059}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5239","reported_seq":5358,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.973038+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:49.973038+0000","last_peered":"2024-09-11T01:15:49.973038+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:49.973038+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5239,"log_dups_size":0,"ondisk_log_size":5239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21450752,"num_objects":5237,"num_object_clones":0,"num_object_copies":10474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1929,"num_write_kb":7716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5327","reported_seq":5451,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.009802+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:50.009802+0000","last_peered":"2024-09-11T01:15:50.009802+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:50.009802+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5327,"log_dups_size":0,"ondisk_log_size":5327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21786624,"num_objects":5319,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1720,"num_write_kb":6880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1127},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5275","reported_seq":5398,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:50.020425+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:50.020425+0000","last_peered":"2024-09-11T01:15:50.020425+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:50.020425+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5275,"log_dups_size":0,"ondisk_log_size":5275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21590016,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1810,"num_write_kb":7240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1123},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5181","reported_seq":5306,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.932748+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:49.932748+0000","last_peered":"2024-09-11T01:15:49.932748+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:49.932748+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5181,"log_dups_size":0,"ondisk_log_size":5181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21173190,"num_objects":5171,"num_object_clones":0,"num_object_copies":10342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1838,"num_write_kb":7352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1017},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":341025502,"num_objects":83267,"num_object_clones":0,"num_object_copies":166534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3408,"num_objects_unfound":0,"num_objects_dirty":83267,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":42402,"num_write_kb":169608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":2936832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":689442816,"data_stored":689287457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83339,"ondisk_log_size":83339,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460368,"kb_used_data":298176,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93911472,"statfs":{"total":96636764160,"available":96165347328,"internally_reserved":0,"allocated":305332224,"data_stored":305075320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789960,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166792,"kb_used_data":58744,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94205048,"statfs":{"total":96636764160,"available":96465969152,"internally_reserved":0,"allocated":60153856,"data_stored":59845906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482448,"kb_used_data":316160,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93889392,"statfs":{"total":96636764160,"available":96142737408,"internally_reserved":0,"allocated":323747840,"data_stored":323486859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,1,0,1,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":323784704,"data_stored":323748574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60264448,"data_stored":60181349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305393664,"data_stored":305357534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:54.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 78 20 85687 85667 4.28945 2.14453 0.0117521 0.0182031 2024-09-11T01:15:55.325 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 79 20 86272 86252 4.26408 2.28516 0.0299419 0.0183145 2024-09-11T01:15:56.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:15:56.325158+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0184172 2024-09-11T01:15:56.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:15:56.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 80 20 86876 86856 4.24027 2.35938 0.0305204 0.0184172 2024-09-11T01:15:57.018 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:57.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 81 20 87433 87413 4.21478 2.17578 0.0325102 0.0185269 2024-09-11T01:15:57.623 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:57.899 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:57.900 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-11T01:15:56.430708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352391921,"num_objects":86044,"num_object_clones":0,"num_object_copies":172088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":86044,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":45179,"num_write_kb":180710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":844,"num_bytes_recovered":3452947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86117,"ondisk_log_size":86117,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1137312,"kb_used_data":686896,"kb_used_omap":12,"kb_used_meta":450355,"kb_avail":281978208,"statfs":{"total":289910292480,"available":288745684992,"internally_reserved":0,"allocated":703381504,"data_stored":702555669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":461163527},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,8,12,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34750464,"num_objects":8484,"num_object_clones":0,"num_object_copies":16968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-367,"num_objects_unfound":0,"num_objects_dirty":8484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8484,"num_write_kb":33936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004066"},"pg_stats":[{"pgid":"2.f","version":"39'5359","reported_seq":5422,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.997099+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:54.997099+0000","last_peered":"2024-09-11T01:15:54.997099+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:54.997099+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950464,"num_objects":5359,"num_object_clones":0,"num_object_copies":10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":21436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1236},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5394","reported_seq":5464,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.766715+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:54.766715+0000","last_peered":"2024-09-11T01:15:54.766715+0000","last_clean":"2024-09-11T01:15:54.766715+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:54.766715+0000","last_undegraded":"2024-09-11T01:15:54.766715+0000","last_fullsized":"2024-09-11T01:15:54.766715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22070214,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3492,"num_write_kb":13968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5262","reported_seq":5335,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.764289+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:54.764289+0000","last_peered":"2024-09-11T01:15:54.764289+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:54.764289+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5262,"log_dups_size":0,"ondisk_log_size":5262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21536768,"num_objects":5258,"num_object_clones":0,"num_object_copies":10516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3323,"num_write_kb":13292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5330","reported_seq":5365,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.775251+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:54.775251+0000","last_peered":"2024-09-11T01:15:54.775251+0000","last_clean":"2024-09-11T01:15:54.775251+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:54.775251+0000","last_undegraded":"2024-09-11T01:15:54.775251+0000","last_fullsized":"2024-09-11T01:15:54.775251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5330,"num_write_kb":21320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5361","reported_seq":5472,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:55.006841+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:55.006841+0000","last_peered":"2024-09-11T01:15:55.006841+0000","last_clean":"2024-09-11T01:15:55.006841+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:55.006841+0000","last_undegraded":"2024-09-11T01:15:55.006841+0000","last_fullsized":"2024-09-11T01:15:55.006841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21930467,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2159,"num_write_kb":8636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5334","reported_seq":5458,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.991534+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:54.991534+0000","last_peered":"2024-09-11T01:15:54.991534+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:54.991534+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21828067,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1215},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5553","reported_seq":5676,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.917320+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:54.917320+0000","last_peered":"2024-09-11T01:15:54.917320+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:54.917320+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5553,"log_dups_size":0,"ondisk_log_size":5553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22721478,"num_objects":5549,"num_object_clones":0,"num_object_copies":11098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1266},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5349","reported_seq":5458,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.983035+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:54.983035+0000","last_peered":"2024-09-11T01:15:54.983035+0000","last_clean":"2024-09-11T01:15:54.983035+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:54.983035+0000","last_undegraded":"2024-09-11T01:15:54.983035+0000","last_fullsized":"2024-09-11T01:15:54.983035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5333","reported_seq":7087,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:54.782209+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:54.782209+0000","last_peered":"2024-09-11T01:15:54.782209+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:54.782209+0000","last_undegraded":"2024-09-11T01:15:54.782209+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5333,"log_dups_size":0,"ondisk_log_size":5333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21815779,"num_objects":5327,"num_object_clones":0,"num_object_copies":10654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":5327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":3448832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3282},{"shards":"2","objects":2045}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5428","reported_seq":5547,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.958681+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:54.958681+0000","last_peered":"2024-09-11T01:15:54.958681+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:54.958681+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1792,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1315},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5269","reported_seq":5353,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.785652+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:54.785652+0000","last_peered":"2024-09-11T01:15:54.785652+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:54.785652+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561827,"num_objects":5265,"num_object_clones":0,"num_object_copies":10530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1238}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5408","reported_seq":5491,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.707999+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:54.707999+0000","last_peered":"2024-09-11T01:15:54.707999+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:54.707999+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5408,"log_dups_size":0,"ondisk_log_size":5408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22126592,"num_objects":5402,"num_object_clones":0,"num_object_copies":10804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3045,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5418","reported_seq":5538,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.996813+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:54.996813+0000","last_peered":"2024-09-11T01:15:54.996813+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:54.996813+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5418,"log_dups_size":0,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22183936,"num_objects":5416,"num_object_clones":0,"num_object_copies":10832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2108,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5509","reported_seq":5633,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:55.021595+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:55.021595+0000","last_peered":"2024-09-11T01:15:55.021595+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:55.021595+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5448","reported_seq":5571,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:55.014982+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:55.014982+0000","last_peered":"2024-09-11T01:15:55.014982+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:55.014982+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5448,"log_dups_size":0,"ondisk_log_size":5448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22298624,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1983,"num_write_kb":7932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1296},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5359","reported_seq":5484,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.999375+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:54.999375+0000","last_peered":"2024-09-11T01:15:54.999375+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:54.999375+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21902278,"num_objects":5349,"num_object_clones":0,"num_object_copies":10698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5349,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2016,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1195},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352391902,"num_objects":86042,"num_object_clones":0,"num_object_copies":172084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":86042,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":45177,"num_write_kb":180708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":703758336,"data_stored":703602977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86114,"ondisk_log_size":86114,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469860,"kb_used_data":303508,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93901980,"statfs":{"total":96636764160,"available":96155627520,"internally_reserved":0,"allocated":310792192,"data_stored":310535288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789960,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":166792,"kb_used_data":58744,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94205048,"statfs":{"total":96636764160,"available":96465969152,"internally_reserved":0,"allocated":60153856,"data_stored":59845906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500660,"kb_used_data":324644,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93871180,"statfs":{"total":96636764160,"available":96124088320,"internally_reserved":0,"allocated":332435456,"data_stored":332174475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,7,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332414976,"data_stored":332378846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":60264448,"data_stored":60181349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311078912,"data_stored":311042782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:57.901 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:58.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 82 20 87966 87946 4.18877 2.08203 0.012393 0.0186408 2024-09-11T01:15:58.477 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:58.480 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:58.507 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-11T01:15:58.431162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352391921,"num_objects":86044,"num_object_clones":0,"num_object_copies":172088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":86044,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":45179,"num_write_kb":180710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":844,"num_bytes_recovered":3452947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86117,"ondisk_log_size":86117,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1150320,"kb_used_data":695744,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281965200,"statfs":{"total":289910292480,"available":288732364800,"internally_reserved":0,"allocated":712441856,"data_stored":711616021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":465423367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,8,12,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5623,"num_write_kb":22492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002570"},"pg_stats":[{"pgid":"2.f","version":"39'5359","reported_seq":5422,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.997099+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:54.997099+0000","last_peered":"2024-09-11T01:15:54.997099+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:54.997099+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950464,"num_objects":5359,"num_object_clones":0,"num_object_copies":10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":21436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1236},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5394","reported_seq":5464,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.766715+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:54.766715+0000","last_peered":"2024-09-11T01:15:54.766715+0000","last_clean":"2024-09-11T01:15:54.766715+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:54.766715+0000","last_undegraded":"2024-09-11T01:15:54.766715+0000","last_fullsized":"2024-09-11T01:15:54.766715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22070214,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3492,"num_write_kb":13968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5262","reported_seq":5335,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.764289+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:54.764289+0000","last_peered":"2024-09-11T01:15:54.764289+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:54.764289+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5262,"log_dups_size":0,"ondisk_log_size":5262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21536768,"num_objects":5258,"num_object_clones":0,"num_object_copies":10516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3323,"num_write_kb":13292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5330","reported_seq":5365,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.775251+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:54.775251+0000","last_peered":"2024-09-11T01:15:54.775251+0000","last_clean":"2024-09-11T01:15:54.775251+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:54.775251+0000","last_undegraded":"2024-09-11T01:15:54.775251+0000","last_fullsized":"2024-09-11T01:15:54.775251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5330,"num_write_kb":21320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5361","reported_seq":5472,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:55.006841+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:55.006841+0000","last_peered":"2024-09-11T01:15:55.006841+0000","last_clean":"2024-09-11T01:15:55.006841+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:55.006841+0000","last_undegraded":"2024-09-11T01:15:55.006841+0000","last_fullsized":"2024-09-11T01:15:55.006841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21930467,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2159,"num_write_kb":8636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5334","reported_seq":5458,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.991534+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:54.991534+0000","last_peered":"2024-09-11T01:15:54.991534+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:54.991534+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21828067,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1215},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5553","reported_seq":5676,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.917320+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:54.917320+0000","last_peered":"2024-09-11T01:15:54.917320+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:54.917320+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5553,"log_dups_size":0,"ondisk_log_size":5553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22721478,"num_objects":5549,"num_object_clones":0,"num_object_copies":11098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1266},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5349","reported_seq":5458,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.983035+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:54.983035+0000","last_peered":"2024-09-11T01:15:54.983035+0000","last_clean":"2024-09-11T01:15:54.983035+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:54.983035+0000","last_undegraded":"2024-09-11T01:15:54.983035+0000","last_fullsized":"2024-09-11T01:15:54.983035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5333","reported_seq":7087,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:54.782209+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:54.782209+0000","last_peered":"2024-09-11T01:15:54.782209+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:54.782209+0000","last_undegraded":"2024-09-11T01:15:54.782209+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5333,"log_dups_size":0,"ondisk_log_size":5333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21815779,"num_objects":5327,"num_object_clones":0,"num_object_copies":10654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":5327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":3448832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3282},{"shards":"2","objects":2045}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5428","reported_seq":5547,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.958681+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:54.958681+0000","last_peered":"2024-09-11T01:15:54.958681+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:54.958681+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1792,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1315},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5269","reported_seq":5353,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.785652+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:54.785652+0000","last_peered":"2024-09-11T01:15:54.785652+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:54.785652+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561827,"num_objects":5265,"num_object_clones":0,"num_object_copies":10530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1238}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5408","reported_seq":5491,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.707999+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:54.707999+0000","last_peered":"2024-09-11T01:15:54.707999+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:54.707999+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5408,"log_dups_size":0,"ondisk_log_size":5408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22126592,"num_objects":5402,"num_object_clones":0,"num_object_copies":10804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3045,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5418","reported_seq":5538,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.996813+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:54.996813+0000","last_peered":"2024-09-11T01:15:54.996813+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:54.996813+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5418,"log_dups_size":0,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22183936,"num_objects":5416,"num_object_clones":0,"num_object_copies":10832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2108,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5509","reported_seq":5633,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:55.021595+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:55.021595+0000","last_peered":"2024-09-11T01:15:55.021595+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:55.021595+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5448","reported_seq":5571,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:55.014982+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:55.014982+0000","last_peered":"2024-09-11T01:15:55.014982+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:55.014982+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5448,"log_dups_size":0,"ondisk_log_size":5448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22298624,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1983,"num_write_kb":7932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1296},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5359","reported_seq":5484,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.999375+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:54.999375+0000","last_peered":"2024-09-11T01:15:54.999375+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:54.999375+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21902278,"num_objects":5349,"num_object_clones":0,"num_object_copies":10698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5349,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2016,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1195},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352391902,"num_objects":86042,"num_object_clones":0,"num_object_copies":172084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":86042,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":45177,"num_write_kb":180708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":713031680,"data_stored":712876321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86114,"ondisk_log_size":86114,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469860,"kb_used_data":303508,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93901980,"statfs":{"total":96636764160,"available":96155627520,"internally_reserved":0,"allocated":310792192,"data_stored":310535288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789961,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179800,"kb_used_data":67592,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94192040,"statfs":{"total":96636764160,"available":96452648960,"internally_reserved":0,"allocated":69214208,"data_stored":68906258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500660,"kb_used_data":324644,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93871180,"statfs":{"total":96636764160,"available":96124088320,"internally_reserved":0,"allocated":332435456,"data_stored":332174475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,7,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332414976,"data_stored":332378846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69537792,"data_stored":69454693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311078912,"data_stored":311042782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'5359", 'reported_seq': 5422, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.997099+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:15:54.997099+0000', 'last_peered': '2024-09-11T01:15:54.997099+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:15:54.997099+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 5359, 'log_dups_size': 0, 'ondisk_log_size': 5359, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21950464, 'num_objects': 5359, 'num_object_clones': 0, 'num_object_copies': 10718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5359, 'num_write_kb': 21436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1236}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'5262", 'reported_seq': 5335, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.764289+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:54.764289+0000', 'last_peered': '2024-09-11T01:15:54.764289+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:54.764289+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 5262, 'log_dups_size': 0, 'ondisk_log_size': 5262, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21536768, 'num_objects': 5258, 'num_object_clones': 0, 'num_object_copies': 10516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5258, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3323, 'num_write_kb': 13292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 1253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'5334", 'reported_seq': 5458, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.991534+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:54.991534+0000', 'last_peered': '2024-09-11T01:15:54.991534+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:54.991534+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 5334, 'log_dups_size': 0, 'ondisk_log_size': 5334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21828067, 'num_objects': 5330, 'num_object_clones': 0, 'num_object_copies': 10660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5330, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1822, 'num_write_kb': 7288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1215}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:58.509 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'5553", 'reported_seq': 5676, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.917320+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:15:54.917320+0000', 'last_peered': '2024-09-11T01:15:54.917320+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:15:54.917320+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 5553, 'log_dups_size': 0, 'ondisk_log_size': 5553, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22721478, 'num_objects': 5549, 'num_object_clones': 0, 'num_object_copies': 11098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5549, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2306, 'num_write_kb': 9224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1266}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'5333", 'reported_seq': 7087, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:54.782209+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:54.782209+0000', 'last_peered': '2024-09-11T01:15:54.782209+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:54.782209+0000', 'last_undegraded': '2024-09-11T01:15:54.782209+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 5333, 'log_dups_size': 0, 'ondisk_log_size': 5333, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21815779, 'num_objects': 5327, 'num_object_clones': 0, 'num_object_copies': 10654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3282, 'num_objects_unfound': 0, 'num_objects_dirty': 5327, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3200, 'num_write_kb': 12800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 842, 'num_bytes_recovered': 3448832, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3282}, {'shards': '2', 'objects': 2045}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'5428", 'reported_seq': 5547, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.958681+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:15:54.958681+0000', 'last_peered': '2024-09-11T01:15:54.958681+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:15:54.958681+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 5428, 'log_dups_size': 0, 'ondisk_log_size': 5428, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22200320, 'num_objects': 5420, 'num_object_clones': 0, 'num_object_copies': 10840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1792, 'num_write_kb': 7168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1315}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'5269", 'reported_seq': 5353, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.785652+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:54.785652+0000', 'last_peered': '2024-09-11T01:15:54.785652+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:54.785652+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 5269, 'log_dups_size': 0, 'ondisk_log_size': 5269, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21561827, 'num_objects': 5265, 'num_object_clones': 0, 'num_object_copies': 10530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5265, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3038, 'num_write_kb': 12152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 1238}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'5408", 'reported_seq': 5491, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.707999+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:54.707999+0000', 'last_peered': '2024-09-11T01:15:54.707999+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:54.707999+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 5408, 'log_dups_size': 0, 'ondisk_log_size': 5408, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22126592, 'num_objects': 5402, 'num_object_clones': 0, 'num_object_copies': 10804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3045, 'num_write_kb': 12180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 1225}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'5418", 'reported_seq': 5538, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.996813+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:15:54.996813+0000', 'last_peered': '2024-09-11T01:15:54.996813+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:15:54.996813+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 5418, 'log_dups_size': 0, 'ondisk_log_size': 5418, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22183936, 'num_objects': 5416, 'num_object_clones': 0, 'num_object_copies': 10832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2108, 'num_write_kb': 8432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1252}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.510 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:58.511 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'5509", 'reported_seq': 5633, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:55.021595+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:15:55.021595+0000', 'last_peered': '2024-09-11T01:15:55.021595+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:15:55.021595+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 5509, 'log_dups_size': 0, 'ondisk_log_size': 5509, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22532096, 'num_objects': 5501, 'num_object_clones': 0, 'num_object_copies': 11002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1902, 'num_write_kb': 7608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1309}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.511 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:58.511 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'5448", 'reported_seq': 5571, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:55.014982+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:55.014982+0000', 'last_peered': '2024-09-11T01:15:55.014982+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:55.014982+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 5448, 'log_dups_size': 0, 'ondisk_log_size': 5448, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22298624, 'num_objects': 5444, 'num_object_clones': 0, 'num_object_copies': 10888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1983, 'num_write_kb': 7932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1296}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.511 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:58.511 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'5359", 'reported_seq': 5484, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.999375+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:15:54.999375+0000', 'last_peered': '2024-09-11T01:15:54.999375+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:15:54.999375+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 5359, 'log_dups_size': 0, 'ondisk_log_size': 5359, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21902278, 'num_objects': 5349, 'num_object_clones': 0, 'num_object_copies': 10698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5349, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2016, 'num_write_kb': 8064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1195}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:58.511 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:59.266 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:59.266 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/172088 objects degraded (26.445%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":159,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86044,"data_bytes":352391921,"bytes_used":1177927680,"bytes_avail":288732364800,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":172088,"degraded_ratio":0.26444609734554414,"misplaced_objects":3282,"misplaced_total":172088,"misplaced_ratio":0.019071637766724003,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83267,"recovering_keys_per_sec":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"write_bytes_sec":1918906,"read_op_per_sec":0,"write_op_per_sec":468},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:15:59.298 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:59.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:59.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 83 20 88495 88475 4.1632 2.06641 0.0595278 0.0187578 2024-09-11T01:15:59.892 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:15:59.893 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:15:59.915 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-11T01:15:58.431162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352391921,"num_objects":86044,"num_object_clones":0,"num_object_copies":172088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":86044,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":45179,"num_write_kb":180710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":844,"num_bytes_recovered":3452947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86117,"ondisk_log_size":86117,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1150320,"kb_used_data":695744,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281965200,"statfs":{"total":289910292480,"available":288732364800,"internally_reserved":0,"allocated":712441856,"data_stored":711616021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":465423367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,8,12,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23031808,"num_objects":5623,"num_object_clones":0,"num_object_copies":11246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5623,"num_write_kb":22492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002570"},"pg_stats":[{"pgid":"2.f","version":"39'5359","reported_seq":5422,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.997099+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:15:54.997099+0000","last_peered":"2024-09-11T01:15:54.997099+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:15:54.997099+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21950464,"num_objects":5359,"num_object_clones":0,"num_object_copies":10718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5359,"num_write_kb":21436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1236},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5394","reported_seq":5464,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.766715+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:54.766715+0000","last_peered":"2024-09-11T01:15:54.766715+0000","last_clean":"2024-09-11T01:15:54.766715+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:54.766715+0000","last_undegraded":"2024-09-11T01:15:54.766715+0000","last_fullsized":"2024-09-11T01:15:54.766715+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5394,"log_dups_size":0,"ondisk_log_size":5394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22070214,"num_objects":5390,"num_object_clones":0,"num_object_copies":10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5390,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3492,"num_write_kb":13968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5262","reported_seq":5335,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.764289+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:54.764289+0000","last_peered":"2024-09-11T01:15:54.764289+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:54.764289+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5262,"log_dups_size":0,"ondisk_log_size":5262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21536768,"num_objects":5258,"num_object_clones":0,"num_object_copies":10516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3323,"num_write_kb":13292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5330","reported_seq":5365,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.775251+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:54.775251+0000","last_peered":"2024-09-11T01:15:54.775251+0000","last_clean":"2024-09-11T01:15:54.775251+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:54.775251+0000","last_undegraded":"2024-09-11T01:15:54.775251+0000","last_fullsized":"2024-09-11T01:15:54.775251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5330,"num_write_kb":21320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5361","reported_seq":5472,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:55.006841+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:55.006841+0000","last_peered":"2024-09-11T01:15:55.006841+0000","last_clean":"2024-09-11T01:15:55.006841+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:55.006841+0000","last_undegraded":"2024-09-11T01:15:55.006841+0000","last_fullsized":"2024-09-11T01:15:55.006841+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21930467,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2159,"num_write_kb":8636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5334","reported_seq":5458,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.991534+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:54.991534+0000","last_peered":"2024-09-11T01:15:54.991534+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:54.991534+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5334,"log_dups_size":0,"ondisk_log_size":5334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21828067,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1822,"num_write_kb":7288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1215},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5553","reported_seq":5676,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.917320+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:15:54.917320+0000","last_peered":"2024-09-11T01:15:54.917320+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:15:54.917320+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5553,"log_dups_size":0,"ondisk_log_size":5553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22721478,"num_objects":5549,"num_object_clones":0,"num_object_copies":11098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2306,"num_write_kb":9224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1266},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5349","reported_seq":5458,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.983035+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:15:54.983035+0000","last_peered":"2024-09-11T01:15:54.983035+0000","last_clean":"2024-09-11T01:15:54.983035+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:15:54.983035+0000","last_undegraded":"2024-09-11T01:15:54.983035+0000","last_fullsized":"2024-09-11T01:15:54.983035+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5349,"log_dups_size":0,"ondisk_log_size":5349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2302,"num_write_kb":9208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5333","reported_seq":7087,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:54.782209+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:54.782209+0000","last_peered":"2024-09-11T01:15:54.782209+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:54.782209+0000","last_undegraded":"2024-09-11T01:15:54.782209+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5333,"log_dups_size":0,"ondisk_log_size":5333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21815779,"num_objects":5327,"num_object_clones":0,"num_object_copies":10654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":5327,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":842,"num_bytes_recovered":3448832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3282},{"shards":"2","objects":2045}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5428","reported_seq":5547,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.958681+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:15:54.958681+0000","last_peered":"2024-09-11T01:15:54.958681+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:15:54.958681+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22200320,"num_objects":5420,"num_object_clones":0,"num_object_copies":10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1792,"num_write_kb":7168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1315},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5269","reported_seq":5353,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.785652+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:54.785652+0000","last_peered":"2024-09-11T01:15:54.785652+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:54.785652+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5269,"log_dups_size":0,"ondisk_log_size":5269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561827,"num_objects":5265,"num_object_clones":0,"num_object_copies":10530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1238}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5408","reported_seq":5491,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.707999+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:54.707999+0000","last_peered":"2024-09-11T01:15:54.707999+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:54.707999+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5408,"log_dups_size":0,"ondisk_log_size":5408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22126592,"num_objects":5402,"num_object_clones":0,"num_object_copies":10804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3045,"num_write_kb":12180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1225}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5418","reported_seq":5538,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.996813+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:15:54.996813+0000","last_peered":"2024-09-11T01:15:54.996813+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:15:54.996813+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5418,"log_dups_size":0,"ondisk_log_size":5418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22183936,"num_objects":5416,"num_object_clones":0,"num_object_copies":10832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2108,"num_write_kb":8432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5509","reported_seq":5633,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:55.021595+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:15:55.021595+0000","last_peered":"2024-09-11T01:15:55.021595+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:15:55.021595+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5509,"log_dups_size":0,"ondisk_log_size":5509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1902,"num_write_kb":7608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5448","reported_seq":5571,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:55.014982+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:55.014982+0000","last_peered":"2024-09-11T01:15:55.014982+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:55.014982+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5448,"log_dups_size":0,"ondisk_log_size":5448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22298624,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1983,"num_write_kb":7932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1296},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5359","reported_seq":5484,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.999375+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:15:54.999375+0000","last_peered":"2024-09-11T01:15:54.999375+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:15:54.999375+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21902278,"num_objects":5349,"num_object_clones":0,"num_object_copies":10698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5349,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2016,"num_write_kb":8064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1195},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352391902,"num_objects":86042,"num_object_clones":0,"num_object_copies":172084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3282,"num_objects_unfound":0,"num_objects_dirty":86042,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":45177,"num_write_kb":180708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":843,"num_bytes_recovered":3452928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":713031680,"data_stored":712876321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86114,"ondisk_log_size":86114,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469860,"kb_used_data":303508,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93901980,"statfs":{"total":96636764160,"available":96155627520,"internally_reserved":0,"allocated":310792192,"data_stored":310535288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":170323630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789961,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":179800,"kb_used_data":67592,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94192040,"statfs":{"total":96636764160,"available":96452648960,"internally_reserved":0,"allocated":69214208,"data_stored":68906258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500660,"kb_used_data":324644,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93871180,"statfs":{"total":96636764160,"available":96124088320,"internally_reserved":0,"allocated":332435456,"data_stored":332174475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,7,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":332414976,"data_stored":332378846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69537792,"data_stored":69454693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":311078912,"data_stored":311042782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:00.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 84 20 89050 89030 4.13945 2.16797 0.0340277 0.0188643 2024-09-11T01:16:01.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 85 20 89557 89537 4.11405 1.98047 0.0211442 0.0189824 2024-09-11T01:16:02.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 86 20 90142 90122 4.09278 2.28516 0.0274084 0.0190815 2024-09-11T01:16:02.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:03.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 87 20 90687 90667 4.07021 2.12891 0.0315246 0.0191864 2024-09-11T01:16:03.487 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:03.488 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:03.515 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-11T01:16:02.432092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363823857,"num_objects":88835,"num_object_clones":0,"num_object_copies":177670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":88835,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":47970,"num_write_kb":191874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":3936275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88908,"ondisk_log_size":88908,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1151780,"kb_used_data":718004,"kb_used_omap":12,"kb_used_meta":433715,"kb_avail":281963740,"statfs":{"total":289910292480,"available":288730869760,"internally_reserved":0,"allocated":735236096,"data_stored":734410261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":444124167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,5,13,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5566,"num_write_kb":22264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002641"},"pg_stats":[{"pgid":"2.f","version":"39'5507","reported_seq":5570,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.000941+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:00.000941+0000","last_peered":"2024-09-11T01:16:00.000941+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:00.000941+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1384},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5566","reported_seq":5636,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.617542+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:59.617542+0000","last_peered":"2024-09-11T01:15:59.617542+0000","last_clean":"2024-09-11T01:15:59.617542+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:59.617542+0000","last_undegraded":"2024-09-11T01:15:59.617542+0000","last_fullsized":"2024-09-11T01:15:59.617542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5566,"log_dups_size":0,"ondisk_log_size":5566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22774726,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5437","reported_seq":5509,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.762182+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:59.762182+0000","last_peered":"2024-09-11T01:15:59.762182+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:59.762182+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5437,"log_dups_size":0,"ondisk_log_size":5437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253568,"num_objects":5433,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3498,"num_write_kb":13992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5505","reported_seq":5540,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.788425+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:59.788425+0000","last_peered":"2024-09-11T01:15:59.788425+0000","last_clean":"2024-09-11T01:15:59.788425+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:59.788425+0000","last_undegraded":"2024-09-11T01:15:59.788425+0000","last_fullsized":"2024-09-11T01:15:59.788425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5505,"log_dups_size":0,"ondisk_log_size":5505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5505,"num_write_kb":22020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5542","reported_seq":5651,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.966752+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:59.966752+0000","last_peered":"2024-09-11T01:15:59.966752+0000","last_clean":"2024-09-11T01:15:59.966752+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:59.966752+0000","last_undegraded":"2024-09-11T01:15:59.966752+0000","last_fullsized":"2024-09-11T01:15:59.966752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671843,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2340,"num_write_kb":9360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5517","reported_seq":5641,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.955246+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:59.955246+0000","last_peered":"2024-09-11T01:15:59.955246+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:59.955246+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22577635,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2005,"num_write_kb":8020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1398},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5723","reported_seq":5846,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.037285+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:00.037285+0000","last_peered":"2024-09-11T01:16:00.037285+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:00.037285+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23417798,"num_objects":5719,"num_object_clones":0,"num_object_copies":11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5719,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2476,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1436},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5535","reported_seq":5644,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:00.004638+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:00.004638+0000","last_peered":"2024-09-11T01:16:00.004638+0000","last_clean":"2024-09-11T01:16:00.004638+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:00.004638+0000","last_undegraded":"2024-09-11T01:16:00.004638+0000","last_fullsized":"2024-09-11T01:16:00.004638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5535,"log_dups_size":0,"ondisk_log_size":5535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5534","reported_seq":7526,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:59.788982+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:59.788982+0000","last_peered":"2024-09-11T01:15:59.788982+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:59.788982+0000","last_undegraded":"2024-09-11T01:15:59.788982+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5534,"log_dups_size":0,"ondisk_log_size":5534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22639075,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":3932160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3164},{"shards":"2","objects":2364}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5623","reported_seq":5742,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.020714+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:00.020714+0000","last_peered":"2024-09-11T01:16:00.020714+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:00.020714+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5436","reported_seq":5520,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.703839+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:59.703839+0000","last_peered":"2024-09-11T01:15:59.703839+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:59.703839+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5436,"log_dups_size":0,"ondisk_log_size":5436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245859,"num_objects":5432,"num_object_clones":0,"num_object_copies":10864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5584","reported_seq":5667,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.740707+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:59.740707+0000","last_peered":"2024-09-11T01:15:59.740707+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:59.740707+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5584,"log_dups_size":0,"ondisk_log_size":5584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5586","reported_seq":5704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.000906+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:00.000906+0000","last_peered":"2024-09-11T01:16:00.000906+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:00.000906+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":5584,"num_object_clones":0,"num_object_copies":11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2276,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1420},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5673","reported_seq":5797,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.016710+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:00.016710+0000","last_peered":"2024-09-11T01:16:00.016710+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:00.016710+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5673,"log_dups_size":0,"ondisk_log_size":5673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2066,"num_write_kb":8264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1473},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5617","reported_seq":5740,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.987809+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:59.987809+0000","last_peered":"2024-09-11T01:15:59.987809+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:59.987809+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990848,"num_objects":5613,"num_object_clones":0,"num_object_copies":11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1465},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5520","reported_seq":5645,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.016787+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:00.016787+0000","last_peered":"2024-09-11T01:16:00.016787+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:00.016787+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22561734,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1356},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363823838,"num_objects":88833,"num_object_clones":0,"num_object_copies":177666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":88833,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":47968,"num_write_kb":191872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":3936256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":736169984,"data_stored":736014625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88905,"ondisk_log_size":88905,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463260,"kb_used_data":309388,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93908580,"statfs":{"total":96636764160,"available":96162385920,"internally_reserved":0,"allocated":316813312,"data_stored":316556408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175532,"kb_used_data":75804,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94196308,"statfs":{"total":96636764160,"available":96457019392,"internally_reserved":0,"allocated":77623296,"data_stored":77315346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":102100655},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,5,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512988,"kb_used_data":332812,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93858852,"statfs":{"total":96636764160,"available":96111464448,"internally_reserved":0,"allocated":340799488,"data_stored":340538507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,6,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340881408,"data_stored":340845278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78360576,"data_stored":78277477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316928000,"data_stored":316891870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:03.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:04.090 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:04.091 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:04.125 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-11T01:16:02.432092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363823857,"num_objects":88835,"num_object_clones":0,"num_object_copies":177670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":88835,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":47970,"num_write_kb":191874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":3936275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88908,"ondisk_log_size":88908,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1151780,"kb_used_data":718004,"kb_used_omap":12,"kb_used_meta":433715,"kb_avail":281963740,"statfs":{"total":289910292480,"available":288730869760,"internally_reserved":0,"allocated":735236096,"data_stored":734410261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":444124167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,5,13,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5566,"num_write_kb":22264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002641"},"pg_stats":[{"pgid":"2.f","version":"39'5507","reported_seq":5570,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.000941+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:00.000941+0000","last_peered":"2024-09-11T01:16:00.000941+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:00.000941+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1384},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5566","reported_seq":5636,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.617542+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:59.617542+0000","last_peered":"2024-09-11T01:15:59.617542+0000","last_clean":"2024-09-11T01:15:59.617542+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:59.617542+0000","last_undegraded":"2024-09-11T01:15:59.617542+0000","last_fullsized":"2024-09-11T01:15:59.617542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5566,"log_dups_size":0,"ondisk_log_size":5566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22774726,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5437","reported_seq":5509,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.762182+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:59.762182+0000","last_peered":"2024-09-11T01:15:59.762182+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:59.762182+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5437,"log_dups_size":0,"ondisk_log_size":5437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253568,"num_objects":5433,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3498,"num_write_kb":13992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5505","reported_seq":5540,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.788425+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:59.788425+0000","last_peered":"2024-09-11T01:15:59.788425+0000","last_clean":"2024-09-11T01:15:59.788425+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:59.788425+0000","last_undegraded":"2024-09-11T01:15:59.788425+0000","last_fullsized":"2024-09-11T01:15:59.788425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5505,"log_dups_size":0,"ondisk_log_size":5505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5505,"num_write_kb":22020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5542","reported_seq":5651,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.966752+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:59.966752+0000","last_peered":"2024-09-11T01:15:59.966752+0000","last_clean":"2024-09-11T01:15:59.966752+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:59.966752+0000","last_undegraded":"2024-09-11T01:15:59.966752+0000","last_fullsized":"2024-09-11T01:15:59.966752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671843,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2340,"num_write_kb":9360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5517","reported_seq":5641,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.955246+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:59.955246+0000","last_peered":"2024-09-11T01:15:59.955246+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:59.955246+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22577635,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2005,"num_write_kb":8020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1398},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5723","reported_seq":5846,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.037285+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:00.037285+0000","last_peered":"2024-09-11T01:16:00.037285+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:00.037285+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23417798,"num_objects":5719,"num_object_clones":0,"num_object_copies":11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5719,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2476,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1436},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5535","reported_seq":5644,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:00.004638+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:00.004638+0000","last_peered":"2024-09-11T01:16:00.004638+0000","last_clean":"2024-09-11T01:16:00.004638+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:00.004638+0000","last_undegraded":"2024-09-11T01:16:00.004638+0000","last_fullsized":"2024-09-11T01:16:00.004638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5535,"log_dups_size":0,"ondisk_log_size":5535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5534","reported_seq":7526,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:59.788982+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:59.788982+0000","last_peered":"2024-09-11T01:15:59.788982+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:59.788982+0000","last_undegraded":"2024-09-11T01:15:59.788982+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5534,"log_dups_size":0,"ondisk_log_size":5534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22639075,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":3932160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3164},{"shards":"2","objects":2364}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5623","reported_seq":5742,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.020714+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:00.020714+0000","last_peered":"2024-09-11T01:16:00.020714+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:00.020714+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5436","reported_seq":5520,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.703839+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:59.703839+0000","last_peered":"2024-09-11T01:15:59.703839+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:59.703839+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5436,"log_dups_size":0,"ondisk_log_size":5436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245859,"num_objects":5432,"num_object_clones":0,"num_object_copies":10864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5584","reported_seq":5667,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.740707+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:59.740707+0000","last_peered":"2024-09-11T01:15:59.740707+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:59.740707+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5584,"log_dups_size":0,"ondisk_log_size":5584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5586","reported_seq":5704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.000906+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:00.000906+0000","last_peered":"2024-09-11T01:16:00.000906+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:00.000906+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":5584,"num_object_clones":0,"num_object_copies":11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2276,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1420},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5673","reported_seq":5797,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.016710+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:00.016710+0000","last_peered":"2024-09-11T01:16:00.016710+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:00.016710+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5673,"log_dups_size":0,"ondisk_log_size":5673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2066,"num_write_kb":8264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1473},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5617","reported_seq":5740,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.987809+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:59.987809+0000","last_peered":"2024-09-11T01:15:59.987809+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:59.987809+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990848,"num_objects":5613,"num_object_clones":0,"num_object_copies":11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1465},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5520","reported_seq":5645,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.016787+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:00.016787+0000","last_peered":"2024-09-11T01:16:00.016787+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:00.016787+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22561734,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1356},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363823838,"num_objects":88833,"num_object_clones":0,"num_object_copies":177666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":88833,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":47968,"num_write_kb":191872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":3936256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":736169984,"data_stored":736014625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88905,"ondisk_log_size":88905,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463260,"kb_used_data":309388,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93908580,"statfs":{"total":96636764160,"available":96162385920,"internally_reserved":0,"allocated":316813312,"data_stored":316556408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175532,"kb_used_data":75804,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94196308,"statfs":{"total":96636764160,"available":96457019392,"internally_reserved":0,"allocated":77623296,"data_stored":77315346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":102100655},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,5,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512988,"kb_used_data":332812,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93858852,"statfs":{"total":96636764160,"available":96111464448,"internally_reserved":0,"allocated":340799488,"data_stored":340538507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,6,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340881408,"data_stored":340845278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78360576,"data_stored":78277477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316928000,"data_stored":316891870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:04.126 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:04.126 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'5507", 'reported_seq': 5570, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:00.000941+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:00.000941+0000', 'last_peered': '2024-09-11T01:16:00.000941+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:00.000941+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 5507, 'log_dups_size': 0, 'ondisk_log_size': 5507, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22556672, 'num_objects': 5507, 'num_object_clones': 0, 'num_object_copies': 11014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5507, 'num_write_kb': 22028, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1384}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.126 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:04.126 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'5437", 'reported_seq': 5509, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.762182+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:15:59.762182+0000', 'last_peered': '2024-09-11T01:15:59.762182+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:15:59.762182+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 5437, 'log_dups_size': 0, 'ondisk_log_size': 5437, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22253568, 'num_objects': 5433, 'num_object_clones': 0, 'num_object_copies': 10866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3498, 'num_write_kb': 13992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 1428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.126 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'5517", 'reported_seq': 5641, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.955246+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:15:59.955246+0000', 'last_peered': '2024-09-11T01:15:59.955246+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:15:59.955246+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 5517, 'log_dups_size': 0, 'ondisk_log_size': 5517, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22577635, 'num_objects': 5513, 'num_object_clones': 0, 'num_object_copies': 11026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5513, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2005, 'num_write_kb': 8020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1398}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'5723", 'reported_seq': 5846, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:00.037285+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:00.037285+0000', 'last_peered': '2024-09-11T01:16:00.037285+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:00.037285+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 5723, 'log_dups_size': 0, 'ondisk_log_size': 5723, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23417798, 'num_objects': 5719, 'num_object_clones': 0, 'num_object_copies': 11438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5719, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2476, 'num_write_kb': 9904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1436}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'5534", 'reported_seq': 7526, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:59.788982+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:15:59.788982+0000', 'last_peered': '2024-09-11T01:15:59.788982+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:15:59.788982+0000', 'last_undegraded': '2024-09-11T01:15:59.788982+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 5534, 'log_dups_size': 0, 'ondisk_log_size': 5534, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22639075, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3164, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3401, 'num_write_kb': 13604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 960, 'num_bytes_recovered': 3932160, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3164}, {'shards': '2', 'objects': 2364}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'5623", 'reported_seq': 5742, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:00.020714+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:00.020714+0000', 'last_peered': '2024-09-11T01:16:00.020714+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:00.020714+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 5623, 'log_dups_size': 0, 'ondisk_log_size': 5623, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22999040, 'num_objects': 5615, 'num_object_clones': 0, 'num_object_copies': 11230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1987, 'num_write_kb': 7948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1510}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'5436", 'reported_seq': 5520, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.703839+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:15:59.703839+0000', 'last_peered': '2024-09-11T01:15:59.703839+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:15:59.703839+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 5436, 'log_dups_size': 0, 'ondisk_log_size': 5436, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22245859, 'num_objects': 5432, 'num_object_clones': 0, 'num_object_copies': 10864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5432, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3205, 'num_write_kb': 12820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 1405}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'5584", 'reported_seq': 5667, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.740707+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:15:59.740707+0000', 'last_peered': '2024-09-11T01:15:59.740707+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:15:59.740707+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 5584, 'log_dups_size': 0, 'ondisk_log_size': 5584, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22847488, 'num_objects': 5578, 'num_object_clones': 0, 'num_object_copies': 11156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3221, 'num_write_kb': 12884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 1401}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'5586", 'reported_seq': 5704, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:00.000906+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:00.000906+0000', 'last_peered': '2024-09-11T01:16:00.000906+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:00.000906+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 5586, 'log_dups_size': 0, 'ondisk_log_size': 5586, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22872064, 'num_objects': 5584, 'num_object_clones': 0, 'num_object_copies': 11168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2276, 'num_write_kb': 9104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1420}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'5673", 'reported_seq': 5797, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:00.016710+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:00.016710+0000', 'last_peered': '2024-09-11T01:16:00.016710+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:00.016710+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 5673, 'log_dups_size': 0, 'ondisk_log_size': 5673, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23203840, 'num_objects': 5665, 'num_object_clones': 0, 'num_object_copies': 11330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2066, 'num_write_kb': 8264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1473}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'5617", 'reported_seq': 5740, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.987809+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:15:59.987809+0000', 'last_peered': '2024-09-11T01:15:59.987809+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:15:59.987809+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 5617, 'log_dups_size': 0, 'ondisk_log_size': 5617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22990848, 'num_objects': 5613, 'num_object_clones': 0, 'num_object_copies': 11226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2152, 'num_write_kb': 8608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1465}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:04.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'5520", 'reported_seq': 5645, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:00.016787+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:00.016787+0000', 'last_peered': '2024-09-11T01:16:00.016787+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:00.016787+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 5520, 'log_dups_size': 0, 'ondisk_log_size': 5520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22561734, 'num_objects': 5510, 'num_object_clones': 0, 'num_object_copies': 11020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5510, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2177, 'num_write_kb': 8708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1356}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:04.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 88 20 91190 91170 4.04628 1.96484 0.0191505 0.0193 2024-09-11T01:16:04.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:04.837 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/177670 objects degraded (25.614%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88835,"data_bytes":363823857,"bytes_used":1179422720,"bytes_avail":288730869760,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":177670,"degraded_ratio":0.25613778353126582,"misplaced_objects":3164,"misplaced_total":177670,"misplaced_ratio":0.01780829627961952,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83266,"recovering_keys_per_sec":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"write_bytes_sec":1899438,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:16:04.863 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:04.863 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:05.326 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 89 20 91695 91675 4.02298 1.97266 0.0266533 0.0194118 2024-09-11T01:16:05.422 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:05.423 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:05.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-11T01:16:04.432564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":363823857,"num_objects":88835,"num_object_clones":0,"num_object_copies":177670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":88835,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":47970,"num_write_kb":191874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":962,"num_bytes_recovered":3936275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88908,"ondisk_log_size":88908,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1151780,"kb_used_data":718004,"kb_used_omap":12,"kb_used_meta":433715,"kb_avail":281963740,"statfs":{"total":289910292480,"available":288730869760,"internally_reserved":0,"allocated":735236096,"data_stored":734410261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":444124167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,5,13,4,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22798336,"num_objects":5566,"num_object_clones":0,"num_object_copies":11132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5566,"num_write_kb":22264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002669"},"pg_stats":[{"pgid":"2.f","version":"39'5507","reported_seq":5570,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.000941+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:00.000941+0000","last_peered":"2024-09-11T01:16:00.000941+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:00.000941+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22556672,"num_objects":5507,"num_object_clones":0,"num_object_copies":11014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5507,"num_write_kb":22028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1384},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5566","reported_seq":5636,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.617542+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:15:59.617542+0000","last_peered":"2024-09-11T01:15:59.617542+0000","last_clean":"2024-09-11T01:15:59.617542+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:15:59.617542+0000","last_undegraded":"2024-09-11T01:15:59.617542+0000","last_fullsized":"2024-09-11T01:15:59.617542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5566,"log_dups_size":0,"ondisk_log_size":5566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22774726,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5437","reported_seq":5509,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.762182+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:15:59.762182+0000","last_peered":"2024-09-11T01:15:59.762182+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:15:59.762182+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5437,"log_dups_size":0,"ondisk_log_size":5437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22253568,"num_objects":5433,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3498,"num_write_kb":13992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1428}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5505","reported_seq":5540,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.788425+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:15:59.788425+0000","last_peered":"2024-09-11T01:15:59.788425+0000","last_clean":"2024-09-11T01:15:59.788425+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:15:59.788425+0000","last_undegraded":"2024-09-11T01:15:59.788425+0000","last_fullsized":"2024-09-11T01:15:59.788425+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5505,"log_dups_size":0,"ondisk_log_size":5505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5505,"num_write_kb":22020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5542","reported_seq":5651,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.966752+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:15:59.966752+0000","last_peered":"2024-09-11T01:15:59.966752+0000","last_clean":"2024-09-11T01:15:59.966752+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:15:59.966752+0000","last_undegraded":"2024-09-11T01:15:59.966752+0000","last_fullsized":"2024-09-11T01:15:59.966752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671843,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2340,"num_write_kb":9360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5517","reported_seq":5641,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.955246+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:15:59.955246+0000","last_peered":"2024-09-11T01:15:59.955246+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:15:59.955246+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5517,"log_dups_size":0,"ondisk_log_size":5517,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22577635,"num_objects":5513,"num_object_clones":0,"num_object_copies":11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2005,"num_write_kb":8020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1398},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5723","reported_seq":5846,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.037285+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:00.037285+0000","last_peered":"2024-09-11T01:16:00.037285+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:00.037285+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5723,"log_dups_size":0,"ondisk_log_size":5723,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23417798,"num_objects":5719,"num_object_clones":0,"num_object_copies":11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5719,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2476,"num_write_kb":9904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1436},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5535","reported_seq":5644,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:00.004638+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:00.004638+0000","last_peered":"2024-09-11T01:16:00.004638+0000","last_clean":"2024-09-11T01:16:00.004638+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:00.004638+0000","last_undegraded":"2024-09-11T01:16:00.004638+0000","last_fullsized":"2024-09-11T01:16:00.004638+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5535,"log_dups_size":0,"ondisk_log_size":5535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2488,"num_write_kb":9952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5534","reported_seq":7526,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:59.788982+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:15:59.788982+0000","last_peered":"2024-09-11T01:15:59.788982+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:15:59.788982+0000","last_undegraded":"2024-09-11T01:15:59.788982+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5534,"log_dups_size":0,"ondisk_log_size":5534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22639075,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3401,"num_write_kb":13604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":3932160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3164},{"shards":"2","objects":2364}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5623","reported_seq":5742,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.020714+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:00.020714+0000","last_peered":"2024-09-11T01:16:00.020714+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:00.020714+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5623,"log_dups_size":0,"ondisk_log_size":5623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22999040,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1987,"num_write_kb":7948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5436","reported_seq":5520,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.703839+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:15:59.703839+0000","last_peered":"2024-09-11T01:15:59.703839+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:15:59.703839+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5436,"log_dups_size":0,"ondisk_log_size":5436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245859,"num_objects":5432,"num_object_clones":0,"num_object_copies":10864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3205,"num_write_kb":12820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5584","reported_seq":5667,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.740707+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:15:59.740707+0000","last_peered":"2024-09-11T01:15:59.740707+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:15:59.740707+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5584,"log_dups_size":0,"ondisk_log_size":5584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1401}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5586","reported_seq":5704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.000906+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:00.000906+0000","last_peered":"2024-09-11T01:16:00.000906+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:00.000906+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5586,"log_dups_size":0,"ondisk_log_size":5586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":5584,"num_object_clones":0,"num_object_copies":11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2276,"num_write_kb":9104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1420},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5673","reported_seq":5797,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.016710+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:00.016710+0000","last_peered":"2024-09-11T01:16:00.016710+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:00.016710+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5673,"log_dups_size":0,"ondisk_log_size":5673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2066,"num_write_kb":8264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1473},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5617","reported_seq":5740,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.987809+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:15:59.987809+0000","last_peered":"2024-09-11T01:15:59.987809+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:15:59.987809+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22990848,"num_objects":5613,"num_object_clones":0,"num_object_copies":11226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2152,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1465},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5520","reported_seq":5645,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:00.016787+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:00.016787+0000","last_peered":"2024-09-11T01:16:00.016787+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:00.016787+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5520,"log_dups_size":0,"ondisk_log_size":5520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22561734,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1356},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":363823838,"num_objects":88833,"num_object_clones":0,"num_object_copies":177666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3164,"num_objects_unfound":0,"num_objects_dirty":88833,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":47968,"num_write_kb":191872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":3936256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":736169984,"data_stored":736014625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88905,"ondisk_log_size":88905,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463260,"kb_used_data":309388,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93908580,"statfs":{"total":96636764160,"available":96162385920,"internally_reserved":0,"allocated":316813312,"data_stored":316556408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":157544110},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":175532,"kb_used_data":75804,"kb_used_omap":4,"kb_used_meta":99707,"kb_avail":94196308,"statfs":{"total":96636764160,"available":96457019392,"internally_reserved":0,"allocated":77623296,"data_stored":77315346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":102100655},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,5,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512988,"kb_used_data":332812,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93858852,"statfs":{"total":96636764160,"available":96111464448,"internally_reserved":0,"allocated":340799488,"data_stored":340538507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,6,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":340881408,"data_stored":340845278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78360576,"data_stored":78277477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316928000,"data_stored":316891870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:06.329 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 90 20 92259 92239 4.00276 2.20312 0.0739103 0.019508 2024-09-11T01:16:07.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 91 20 92835 92815 3.9835 2.25 0.0211369 0.0196036 2024-09-11T01:16:08.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 92 20 93412 93392 3.9647 2.25391 0.0284649 0.0196978 2024-09-11T01:16:08.455 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:09.012 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:09.020 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:09.040 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-11T01:16:08.433433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374768369,"num_objects":91507,"num_object_clones":0,"num_object_copies":183014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3037,"num_objects_unfound":0,"num_objects_dirty":91507,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":50642,"num_write_kb":202562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1089,"num_bytes_recovered":4456467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91580,"ondisk_log_size":91580,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1208504,"kb_used_data":740040,"kb_used_omap":12,"kb_used_meta":468403,"kb_avail":281907016,"statfs":{"total":289910292480,"available":288672784384,"internally_reserved":0,"allocated":757800960,"data_stored":756975125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":479644679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,9,4,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22376448,"num_objects":5463,"num_object_clones":0,"num_object_copies":10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002724"},"pg_stats":[{"pgid":"2.f","version":"39'5675","reported_seq":5738,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.042492+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:05.042492+0000","last_peered":"2024-09-11T01:16:05.042492+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:05.042492+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5675,"log_dups_size":0,"ondisk_log_size":5675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23244800,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1552},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5738","reported_seq":5808,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.733644+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:04.733644+0000","last_peered":"2024-09-11T01:16:04.733644+0000","last_clean":"2024-09-11T01:16:04.733644+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:04.733644+0000","last_undegraded":"2024-09-11T01:16:04.733644+0000","last_fullsized":"2024-09-11T01:16:04.733644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23479238,"num_objects":5734,"num_object_clones":0,"num_object_copies":11468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5734,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3836,"num_write_kb":15344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5622","reported_seq":5695,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.719682+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:04.719682+0000","last_peered":"2024-09-11T01:16:04.719682+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:04.719682+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5622,"log_dups_size":0,"ondisk_log_size":5622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23011328,"num_objects":5618,"num_object_clones":0,"num_object_copies":11236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3683,"num_write_kb":14732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5672","reported_seq":5707,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.817517+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:04.817517+0000","last_peered":"2024-09-11T01:16:04.817517+0000","last_clean":"2024-09-11T01:16:04.817517+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:04.817517+0000","last_undegraded":"2024-09-11T01:16:04.817517+0000","last_fullsized":"2024-09-11T01:16:04.817517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5672,"log_dups_size":0,"ondisk_log_size":5672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5672,"num_write_kb":22688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5693","reported_seq":5804,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.955327+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:04.955327+0000","last_peered":"2024-09-11T01:16:04.955327+0000","last_clean":"2024-09-11T01:16:04.955327+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:04.955327+0000","last_undegraded":"2024-09-11T01:16:04.955327+0000","last_fullsized":"2024-09-11T01:16:04.955327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23290339,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2491,"num_write_kb":9964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5689","reported_seq":5813,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.981028+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:04.981028+0000","last_peered":"2024-09-11T01:16:04.981028+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:04.981028+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23282147,"num_objects":5685,"num_object_clones":0,"num_object_copies":11370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5685,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1570},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5864","reported_seq":5987,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.000450+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:05.000450+0000","last_peered":"2024-09-11T01:16:05.000450+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:05.000450+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23995334,"num_objects":5860,"num_object_clones":0,"num_object_copies":11720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5860,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2617,"num_write_kb":10468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1577},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5702","reported_seq":5811,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:05.028316+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:05.028316+0000","last_peered":"2024-09-11T01:16:05.028316+0000","last_clean":"2024-09-11T01:16:05.028316+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:05.028316+0000","last_undegraded":"2024-09-11T01:16:05.028316+0000","last_fullsized":"2024-09-11T01:16:05.028316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2655,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5692","reported_seq":7939,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:04.826978+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:04.826978+0000","last_peered":"2024-09-11T01:16:04.826978+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:04.826978+0000","last_undegraded":"2024-09-11T01:16:04.826978+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23286243,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3037,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3559,"num_write_kb":14236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1087,"num_bytes_recovered":4452352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3037},{"shards":"2","objects":2649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5786","reported_seq":5905,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.950286+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:04.950286+0000","last_peered":"2024-09-11T01:16:04.950286+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:04.950286+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1673},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5604","reported_seq":5688,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.780512+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:04.780512+0000","last_peered":"2024-09-11T01:16:04.780512+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:04.780512+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5604,"log_dups_size":0,"ondisk_log_size":5604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933987,"num_objects":5600,"num_object_clones":0,"num_object_copies":11200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3373,"num_write_kb":13492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5776","reported_seq":5859,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.801965+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:04.801965+0000","last_peered":"2024-09-11T01:16:04.801965+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:04.801965+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5776,"log_dups_size":0,"ondisk_log_size":5776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5741","reported_seq":5861,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.071462+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:05.071462+0000","last_peered":"2024-09-11T01:16:05.071462+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:05.071462+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5741,"log_dups_size":0,"ondisk_log_size":5741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23506944,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1575},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5829","reported_seq":5953,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.929767+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:04.929767+0000","last_peered":"2024-09-11T01:16:04.929767+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:04.929767+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5829,"log_dups_size":0,"ondisk_log_size":5829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2222,"num_write_kb":8888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1629},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5793","reported_seq":5916,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.055265+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:05.055265+0000","last_peered":"2024-09-11T01:16:05.055265+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:05.055265+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5793,"log_dups_size":0,"ondisk_log_size":5793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23711744,"num_objects":5789,"num_object_clones":0,"num_object_copies":11578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2328,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1641},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5701","reported_seq":5826,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.067576+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:05.067576+0000","last_peered":"2024-09-11T01:16:05.067576+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:05.067576+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23303110,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2358,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1537},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374768350,"num_objects":91505,"num_object_clones":0,"num_object_copies":183010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3037,"num_objects_unfound":0,"num_objects_dirty":91505,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":50640,"num_write_kb":202560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1088,"num_bytes_recovered":4456448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":758706176,"data_stored":758550817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91577,"ondisk_log_size":91577,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472988,"kb_used_data":314956,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93898852,"statfs":{"total":96636764160,"available":96152424448,"internally_reserved":0,"allocated":322514944,"data_stored":322258040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789963,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":210444,"kb_used_data":84348,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94161396,"statfs":{"total":96636764160,"available":96421269504,"internally_reserved":0,"allocated":86372352,"data_stored":86064402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129101487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525072,"kb_used_data":340736,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93846768,"statfs":{"total":96636764160,"available":96099090432,"internally_reserved":0,"allocated":348913664,"data_stored":348652683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,0,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":348962816,"data_stored":348926686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87199744,"data_stored":87116645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322543616,"data_stored":322507486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:09.042 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:09.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 93 20 93917 93897 3.94328 1.97266 0.0451681 0.0198043 2024-09-11T01:16:09.635 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:09.635 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:09.663 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-11T01:16:08.433433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374768369,"num_objects":91507,"num_object_clones":0,"num_object_copies":183014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3037,"num_objects_unfound":0,"num_objects_dirty":91507,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":50642,"num_write_kb":202562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1089,"num_bytes_recovered":4456467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91580,"ondisk_log_size":91580,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1208504,"kb_used_data":740040,"kb_used_omap":12,"kb_used_meta":468403,"kb_avail":281907016,"statfs":{"total":289910292480,"available":288672784384,"internally_reserved":0,"allocated":757800960,"data_stored":756975125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":479644679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,9,4,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22376448,"num_objects":5463,"num_object_clones":0,"num_object_copies":10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":21852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002724"},"pg_stats":[{"pgid":"2.f","version":"39'5675","reported_seq":5738,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.042492+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:05.042492+0000","last_peered":"2024-09-11T01:16:05.042492+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:05.042492+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5675,"log_dups_size":0,"ondisk_log_size":5675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23244800,"num_objects":5675,"num_object_clones":0,"num_object_copies":11350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5675,"num_write_kb":22700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1552},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5738","reported_seq":5808,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.733644+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:04.733644+0000","last_peered":"2024-09-11T01:16:04.733644+0000","last_clean":"2024-09-11T01:16:04.733644+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:04.733644+0000","last_undegraded":"2024-09-11T01:16:04.733644+0000","last_fullsized":"2024-09-11T01:16:04.733644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23479238,"num_objects":5734,"num_object_clones":0,"num_object_copies":11468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5734,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3836,"num_write_kb":15344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5622","reported_seq":5695,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.719682+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:04.719682+0000","last_peered":"2024-09-11T01:16:04.719682+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:04.719682+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5622,"log_dups_size":0,"ondisk_log_size":5622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23011328,"num_objects":5618,"num_object_clones":0,"num_object_copies":11236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3683,"num_write_kb":14732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1613}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5672","reported_seq":5707,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.817517+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:04.817517+0000","last_peered":"2024-09-11T01:16:04.817517+0000","last_clean":"2024-09-11T01:16:04.817517+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:04.817517+0000","last_undegraded":"2024-09-11T01:16:04.817517+0000","last_fullsized":"2024-09-11T01:16:04.817517+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5672,"log_dups_size":0,"ondisk_log_size":5672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5672,"num_write_kb":22688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5693","reported_seq":5804,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.955327+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:04.955327+0000","last_peered":"2024-09-11T01:16:04.955327+0000","last_clean":"2024-09-11T01:16:04.955327+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:04.955327+0000","last_undegraded":"2024-09-11T01:16:04.955327+0000","last_fullsized":"2024-09-11T01:16:04.955327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5693,"log_dups_size":0,"ondisk_log_size":5693,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23290339,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2491,"num_write_kb":9964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5689","reported_seq":5813,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.981028+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:04.981028+0000","last_peered":"2024-09-11T01:16:04.981028+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:04.981028+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5689,"log_dups_size":0,"ondisk_log_size":5689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23282147,"num_objects":5685,"num_object_clones":0,"num_object_copies":11370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5685,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2177,"num_write_kb":8708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1570},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'5864","reported_seq":5987,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.000450+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:05.000450+0000","last_peered":"2024-09-11T01:16:05.000450+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:05.000450+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":5864,"log_dups_size":0,"ondisk_log_size":5864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23995334,"num_objects":5860,"num_object_clones":0,"num_object_copies":11720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5860,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2617,"num_write_kb":10468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1577},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5702","reported_seq":5811,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:05.028316+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:05.028316+0000","last_peered":"2024-09-11T01:16:05.028316+0000","last_clean":"2024-09-11T01:16:05.028316+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:05.028316+0000","last_undegraded":"2024-09-11T01:16:05.028316+0000","last_fullsized":"2024-09-11T01:16:05.028316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5702,"log_dups_size":0,"ondisk_log_size":5702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2655,"num_write_kb":10620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5692","reported_seq":7939,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:04.826978+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:04.826978+0000","last_peered":"2024-09-11T01:16:04.826978+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:04.826978+0000","last_undegraded":"2024-09-11T01:16:04.826978+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5692,"log_dups_size":0,"ondisk_log_size":5692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23286243,"num_objects":5686,"num_object_clones":0,"num_object_copies":11372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3037,"num_objects_unfound":0,"num_objects_dirty":5686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3559,"num_write_kb":14236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1087,"num_bytes_recovered":4452352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3037},{"shards":"2","objects":2649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5786","reported_seq":5905,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.950286+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:04.950286+0000","last_peered":"2024-09-11T01:16:04.950286+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:04.950286+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5786,"log_dups_size":0,"ondisk_log_size":5786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1673},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5604","reported_seq":5688,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.780512+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:04.780512+0000","last_peered":"2024-09-11T01:16:04.780512+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:04.780512+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5604,"log_dups_size":0,"ondisk_log_size":5604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933987,"num_objects":5600,"num_object_clones":0,"num_object_copies":11200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3373,"num_write_kb":13492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1573}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5776","reported_seq":5859,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.801965+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:04.801965+0000","last_peered":"2024-09-11T01:16:04.801965+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:04.801965+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5776,"log_dups_size":0,"ondisk_log_size":5776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3413,"num_write_kb":13652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5741","reported_seq":5861,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.071462+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:05.071462+0000","last_peered":"2024-09-11T01:16:05.071462+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:05.071462+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5741,"log_dups_size":0,"ondisk_log_size":5741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23506944,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2431,"num_write_kb":9724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1575},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5829","reported_seq":5953,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.929767+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:04.929767+0000","last_peered":"2024-09-11T01:16:04.929767+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:04.929767+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5829,"log_dups_size":0,"ondisk_log_size":5829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23842816,"num_objects":5821,"num_object_clones":0,"num_object_copies":11642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2222,"num_write_kb":8888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1629},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5793","reported_seq":5916,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.055265+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:05.055265+0000","last_peered":"2024-09-11T01:16:05.055265+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:05.055265+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5793,"log_dups_size":0,"ondisk_log_size":5793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23711744,"num_objects":5789,"num_object_clones":0,"num_object_copies":11578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2328,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1641},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5701","reported_seq":5826,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:05.067576+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:05.067576+0000","last_peered":"2024-09-11T01:16:05.067576+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:05.067576+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5701,"log_dups_size":0,"ondisk_log_size":5701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23303110,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2358,"num_write_kb":9432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1537},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374768350,"num_objects":91505,"num_object_clones":0,"num_object_copies":183010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":3037,"num_objects_unfound":0,"num_objects_dirty":91505,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":50640,"num_write_kb":202560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1088,"num_bytes_recovered":4456448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":758706176,"data_stored":758550817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91577,"ondisk_log_size":91577,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472988,"kb_used_data":314956,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93898852,"statfs":{"total":96636764160,"available":96152424448,"internally_reserved":0,"allocated":322514944,"data_stored":322258040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":161803950},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789963,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":210444,"kb_used_data":84348,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94161396,"statfs":{"total":96636764160,"available":96421269504,"internally_reserved":0,"allocated":86372352,"data_stored":86064402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129101487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525072,"kb_used_data":340736,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93846768,"statfs":{"total":96636764160,"available":96099090432,"internally_reserved":0,"allocated":348913664,"data_stored":348652683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,0,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":348962816,"data_stored":348926686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87199744,"data_stored":87116645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":322543616,"data_stored":322507486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:09.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:09.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'5675", 'reported_seq': 5738, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:05.042492+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:05.042492+0000', 'last_peered': '2024-09-11T01:16:05.042492+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:05.042492+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 5675, 'log_dups_size': 0, 'ondisk_log_size': 5675, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23244800, 'num_objects': 5675, 'num_object_clones': 0, 'num_object_copies': 11350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5675, 'num_write_kb': 22700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1552}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:09.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'5622", 'reported_seq': 5695, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:04.719682+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:04.719682+0000', 'last_peered': '2024-09-11T01:16:04.719682+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:04.719682+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 5622, 'log_dups_size': 0, 'ondisk_log_size': 5622, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23011328, 'num_objects': 5618, 'num_object_clones': 0, 'num_object_copies': 11236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5618, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3683, 'num_write_kb': 14732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 1613}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:09.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:09.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'5689", 'reported_seq': 5813, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:04.981028+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:04.981028+0000', 'last_peered': '2024-09-11T01:16:04.981028+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:04.981028+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 5689, 'log_dups_size': 0, 'ondisk_log_size': 5689, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23282147, 'num_objects': 5685, 'num_object_clones': 0, 'num_object_copies': 11370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5685, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2177, 'num_write_kb': 8708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1570}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.666 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:09.666 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'5864", 'reported_seq': 5987, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:05.000450+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:05.000450+0000', 'last_peered': '2024-09-11T01:16:05.000450+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:05.000450+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 5864, 'log_dups_size': 0, 'ondisk_log_size': 5864, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23995334, 'num_objects': 5860, 'num_object_clones': 0, 'num_object_copies': 11720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5860, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2617, 'num_write_kb': 10468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1577}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.666 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:09.666 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'5692", 'reported_seq': 7939, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:04.826978+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:04.826978+0000', 'last_peered': '2024-09-11T01:16:04.826978+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:04.826978+0000', 'last_undegraded': '2024-09-11T01:16:04.826978+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 5692, 'log_dups_size': 0, 'ondisk_log_size': 5692, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23286243, 'num_objects': 5686, 'num_object_clones': 0, 'num_object_copies': 11372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3037, 'num_objects_unfound': 0, 'num_objects_dirty': 5686, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3559, 'num_write_kb': 14236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1087, 'num_bytes_recovered': 4452352, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3037}, {'shards': '2', 'objects': 2649}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:09.666 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:09.666 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'5786", 'reported_seq': 5905, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:04.950286+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:04.950286+0000', 'last_peered': '2024-09-11T01:16:04.950286+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:04.950286+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 5786, 'log_dups_size': 0, 'ondisk_log_size': 5786, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23666688, 'num_objects': 5778, 'num_object_clones': 0, 'num_object_copies': 11556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2150, 'num_write_kb': 8600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1673}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'5604", 'reported_seq': 5688, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:04.780512+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:04.780512+0000', 'last_peered': '2024-09-11T01:16:04.780512+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:04.780512+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 5604, 'log_dups_size': 0, 'ondisk_log_size': 5604, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22933987, 'num_objects': 5600, 'num_object_clones': 0, 'num_object_copies': 11200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5600, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3373, 'num_write_kb': 13492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 1573}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'5776", 'reported_seq': 5859, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:04.801965+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:04.801965+0000', 'last_peered': '2024-09-11T01:16:04.801965+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:04.801965+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 5776, 'log_dups_size': 0, 'ondisk_log_size': 5776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23633920, 'num_objects': 5770, 'num_object_clones': 0, 'num_object_copies': 11540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3413, 'num_write_kb': 13652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 1593}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'5741", 'reported_seq': 5861, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:05.071462+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:05.071462+0000', 'last_peered': '2024-09-11T01:16:05.071462+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:05.071462+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 5741, 'log_dups_size': 0, 'ondisk_log_size': 5741, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23506944, 'num_objects': 5739, 'num_object_clones': 0, 'num_object_copies': 11478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2431, 'num_write_kb': 9724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1575}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'5829", 'reported_seq': 5953, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:04.929767+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:04.929767+0000', 'last_peered': '2024-09-11T01:16:04.929767+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:04.929767+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 5829, 'log_dups_size': 0, 'ondisk_log_size': 5829, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23842816, 'num_objects': 5821, 'num_object_clones': 0, 'num_object_copies': 11642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2222, 'num_write_kb': 8888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1629}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:09.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'5793", 'reported_seq': 5916, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:05.055265+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:05.055265+0000', 'last_peered': '2024-09-11T01:16:05.055265+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:05.055265+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 5793, 'log_dups_size': 0, 'ondisk_log_size': 5793, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23711744, 'num_objects': 5789, 'num_object_clones': 0, 'num_object_copies': 11578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5789, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2328, 'num_write_kb': 9312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1641}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.668 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:09.668 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'5701", 'reported_seq': 5826, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:05.067576+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:05.067576+0000', 'last_peered': '2024-09-11T01:16:05.067576+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:05.067576+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 5701, 'log_dups_size': 0, 'ondisk_log_size': 5701, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23303110, 'num_objects': 5691, 'num_object_clones': 0, 'num_object_copies': 11382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5691, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2358, 'num_write_kb': 9432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1537}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:09.668 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:10.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 94 20 94427 94407 3.92252 1.99219 0.0492606 0.0199088 2024-09-11T01:16:10.407 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:10.407 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/183014 objects degraded (24.866%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":170,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91507,"data_bytes":374768369,"bytes_used":1237508096,"bytes_avail":288672784384,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":183014,"degraded_ratio":0.24865857256821883,"misplaced_objects":3037,"misplaced_total":183014,"misplaced_ratio":0.016594358901504801,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83607,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"write_bytes_sec":1864280,"read_op_per_sec":0,"write_op_per_sec":455},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:16:10.434 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:10.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:11.010 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:11.011 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:11.038 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-11T01:16:10.433868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386085617,"num_objects":94270,"num_object_clones":0,"num_object_copies":188540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":94270,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":53405,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1226,"num_bytes_recovered":5017619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94343,"ondisk_log_size":94343,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1213848,"kb_used_data":753704,"kb_used_omap":12,"kb_used_meta":460083,"kb_avail":281901672,"statfs":{"total":289910292480,"available":288667312128,"internally_reserved":0,"allocated":771792896,"data_stored":770967061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":471124999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,8,1,7,15,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33693696,"num_objects":8226,"num_object_clones":0,"num_object_copies":16452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-382,"num_objects_unfound":0,"num_objects_dirty":8226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8226,"num_write_kb":32904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":382,"num_bytes_recovered":1564672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002706"},"pg_stats":[{"pgid":"2.f","version":"39'5840","reported_seq":5903,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.048517+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:10.048517+0000","last_peered":"2024-09-11T01:16:10.048517+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:10.048517+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5840,"log_dups_size":0,"ondisk_log_size":5840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23920640,"num_objects":5840,"num_object_clones":0,"num_object_copies":11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5840,"num_write_kb":23360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1717},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5900","reported_seq":5970,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.839211+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:09.839211+0000","last_peered":"2024-09-11T01:16:09.839211+0000","last_clean":"2024-09-11T01:16:09.839211+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:09.839211+0000","last_undegraded":"2024-09-11T01:16:09.839211+0000","last_fullsized":"2024-09-11T01:16:09.839211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5900,"log_dups_size":0,"ondisk_log_size":5900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142790,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3998,"num_write_kb":15992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5796","reported_seq":5868,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.815687+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:09.815687+0000","last_peered":"2024-09-11T01:16:09.815687+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:09.815687+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5796,"log_dups_size":0,"ondisk_log_size":5796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23724032,"num_objects":5792,"num_object_clones":0,"num_object_copies":11584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5859","reported_seq":5894,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.827142+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:09.827142+0000","last_peered":"2024-09-11T01:16:09.827142+0000","last_clean":"2024-09-11T01:16:09.827142+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:09.827142+0000","last_undegraded":"2024-09-11T01:16:09.827142+0000","last_fullsized":"2024-09-11T01:16:09.827142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5869","reported_seq":5980,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:10.082422+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:10.082422+0000","last_peered":"2024-09-11T01:16:10.082422+0000","last_clean":"2024-09-11T01:16:10.082422+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:10.082422+0000","last_undegraded":"2024-09-11T01:16:10.082422+0000","last_fullsized":"2024-09-11T01:16:10.082422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24011235,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2667,"num_write_kb":10668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5892","reported_seq":6014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.092137+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:10.092137+0000","last_peered":"2024-09-11T01:16:10.092137+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:10.092137+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113635,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1773},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6044","reported_seq":6167,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.055221+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:10.055221+0000","last_peered":"2024-09-11T01:16:10.055221+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:10.055221+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24732614,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2797,"num_write_kb":11188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1757},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5888","reported_seq":5997,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:10.108047+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:10.108047+0000","last_peered":"2024-09-11T01:16:10.108047+0000","last_clean":"2024-09-11T01:16:10.108047+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:10.108047+0000","last_undegraded":"2024-09-11T01:16:10.108047+0000","last_fullsized":"2024-09-11T01:16:10.108047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109056,"num_objects":5886,"num_object_clones":0,"num_object_copies":11772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2841,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5857","reported_seq":8377,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:09.835817+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:09.835817+0000","last_peered":"2024-09-11T01:16:09.835817+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:09.835817+0000","last_undegraded":"2024-09-11T01:16:09.835817+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5857,"log_dups_size":0,"ondisk_log_size":5857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23962083,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3724,"num_write_kb":14896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":5013504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2900},{"shards":"2","objects":2951}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5943","reported_seq":6062,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.101214+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:10.101214+0000","last_peered":"2024-09-11T01:16:10.101214+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:10.101214+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5788","reported_seq":5872,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.783530+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:09.783530+0000","last_peered":"2024-09-11T01:16:09.783530+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:09.783530+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23687651,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3557,"num_write_kb":14228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1757}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5937","reported_seq":6020,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.803471+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:09.803471+0000","last_peered":"2024-09-11T01:16:09.803471+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:09.803471+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24293376,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3574,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5927","reported_seq":6047,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.082699+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:10.082699+0000","last_peered":"2024-09-11T01:16:10.082699+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:10.082699+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5927,"log_dups_size":0,"ondisk_log_size":5927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24268800,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2617,"num_write_kb":10468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1761},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5970","reported_seq":6094,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.072569+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:10.072569+0000","last_peered":"2024-09-11T01:16:10.072569+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:10.072569+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2363,"num_write_kb":9452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5964","reported_seq":6087,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.936716+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:09.936716+0000","last_peered":"2024-09-11T01:16:09.936716+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:09.936716+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1812},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5866","reported_seq":5991,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.054902+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:10.054902+0000","last_peered":"2024-09-11T01:16:10.054902+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:10.054902+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5866,"log_dups_size":0,"ondisk_log_size":5866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978950,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2523,"num_write_kb":10092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1702},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386085598,"num_objects":94268,"num_object_clones":0,"num_object_copies":188536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":94268,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":53403,"num_write_kb":213612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":5017600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":772861952,"data_stored":772706593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94340,"ondisk_log_size":94340,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482688,"kb_used_data":320496,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93889152,"statfs":{"total":96636764160,"available":96142491648,"internally_reserved":0,"allocated":328187904,"data_stored":327931000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789963,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":210444,"kb_used_data":84348,"kb_used_omap":4,"kb_used_meta":126075,"kb_avail":94161396,"statfs":{"total":96636764160,"available":96421269504,"internally_reserved":0,"allocated":86372352,"data_stored":86064402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129101487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520716,"kb_used_data":348860,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93851124,"statfs":{"total":96636764160,"available":96103550976,"internally_reserved":0,"allocated":357232640,"data_stored":356971659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,0,5,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357441536,"data_stored":357405406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87199744,"data_stored":87116645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328220672,"data_stored":328184542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:11.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 95 20 94954 94934 3.9029 2.05859 0.0477634 0.02001 2024-09-11T01:16:12.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 96 20 95535 95515 3.88589 2.26953 0.0231504 0.0200974 2024-09-11T01:16:13.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 97 20 96104 96084 3.86874 2.22266 0.0686601 0.0201855 2024-09-11T01:16:14.040 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:14.327 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 98 20 96645 96625 3.85083 2.11328 0.0296765 0.0202803 2024-09-11T01:16:14.618 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:14.618 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:14.640 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-11T01:16:14.434807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386085617,"num_objects":94270,"num_object_clones":0,"num_object_copies":188540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":94270,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":53405,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1226,"num_bytes_recovered":5017619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94343,"ondisk_log_size":94343,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1244680,"kb_used_data":762328,"kb_used_omap":12,"kb_used_meta":482291,"kb_avail":281870840,"statfs":{"total":289910292480,"available":288635740160,"internally_reserved":0,"allocated":780623872,"data_stored":779798037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":493865991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,10,1,8,16,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22261760,"num_objects":5435,"num_object_clones":0,"num_object_copies":10870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":5435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5435,"num_write_kb":21740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"2.f","version":"39'5840","reported_seq":5903,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.048517+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:10.048517+0000","last_peered":"2024-09-11T01:16:10.048517+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:10.048517+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5840,"log_dups_size":0,"ondisk_log_size":5840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23920640,"num_objects":5840,"num_object_clones":0,"num_object_copies":11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5840,"num_write_kb":23360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1717},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5900","reported_seq":5970,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.839211+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:09.839211+0000","last_peered":"2024-09-11T01:16:09.839211+0000","last_clean":"2024-09-11T01:16:09.839211+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:09.839211+0000","last_undegraded":"2024-09-11T01:16:09.839211+0000","last_fullsized":"2024-09-11T01:16:09.839211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5900,"log_dups_size":0,"ondisk_log_size":5900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142790,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3998,"num_write_kb":15992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5796","reported_seq":5868,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.815687+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:09.815687+0000","last_peered":"2024-09-11T01:16:09.815687+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:09.815687+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5796,"log_dups_size":0,"ondisk_log_size":5796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23724032,"num_objects":5792,"num_object_clones":0,"num_object_copies":11584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5859","reported_seq":5894,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.827142+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:09.827142+0000","last_peered":"2024-09-11T01:16:09.827142+0000","last_clean":"2024-09-11T01:16:09.827142+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:09.827142+0000","last_undegraded":"2024-09-11T01:16:09.827142+0000","last_fullsized":"2024-09-11T01:16:09.827142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5869","reported_seq":5980,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:10.082422+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:10.082422+0000","last_peered":"2024-09-11T01:16:10.082422+0000","last_clean":"2024-09-11T01:16:10.082422+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:10.082422+0000","last_undegraded":"2024-09-11T01:16:10.082422+0000","last_fullsized":"2024-09-11T01:16:10.082422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24011235,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2667,"num_write_kb":10668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5892","reported_seq":6014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.092137+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:10.092137+0000","last_peered":"2024-09-11T01:16:10.092137+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:10.092137+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113635,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1773},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6044","reported_seq":6167,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.055221+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:10.055221+0000","last_peered":"2024-09-11T01:16:10.055221+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:10.055221+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24732614,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2797,"num_write_kb":11188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1757},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5888","reported_seq":5997,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:10.108047+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:10.108047+0000","last_peered":"2024-09-11T01:16:10.108047+0000","last_clean":"2024-09-11T01:16:10.108047+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:10.108047+0000","last_undegraded":"2024-09-11T01:16:10.108047+0000","last_fullsized":"2024-09-11T01:16:10.108047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109056,"num_objects":5886,"num_object_clones":0,"num_object_copies":11772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2841,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5857","reported_seq":8377,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:09.835817+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:09.835817+0000","last_peered":"2024-09-11T01:16:09.835817+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:09.835817+0000","last_undegraded":"2024-09-11T01:16:09.835817+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5857,"log_dups_size":0,"ondisk_log_size":5857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23962083,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3724,"num_write_kb":14896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":5013504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2900},{"shards":"2","objects":2951}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5943","reported_seq":6062,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.101214+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:10.101214+0000","last_peered":"2024-09-11T01:16:10.101214+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:10.101214+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5788","reported_seq":5872,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.783530+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:09.783530+0000","last_peered":"2024-09-11T01:16:09.783530+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:09.783530+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23687651,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3557,"num_write_kb":14228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1757}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5937","reported_seq":6020,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.803471+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:09.803471+0000","last_peered":"2024-09-11T01:16:09.803471+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:09.803471+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24293376,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3574,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5927","reported_seq":6047,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.082699+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:10.082699+0000","last_peered":"2024-09-11T01:16:10.082699+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:10.082699+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5927,"log_dups_size":0,"ondisk_log_size":5927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24268800,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2617,"num_write_kb":10468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1761},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5970","reported_seq":6094,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.072569+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:10.072569+0000","last_peered":"2024-09-11T01:16:10.072569+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:10.072569+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2363,"num_write_kb":9452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5964","reported_seq":6087,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.936716+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:09.936716+0000","last_peered":"2024-09-11T01:16:09.936716+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:09.936716+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1812},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5866","reported_seq":5991,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.054902+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:10.054902+0000","last_peered":"2024-09-11T01:16:10.054902+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:10.054902+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5866,"log_dups_size":0,"ondisk_log_size":5866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978950,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2523,"num_write_kb":10092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1702},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386085598,"num_objects":94268,"num_object_clones":0,"num_object_copies":188536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":94268,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":53403,"num_write_kb":213612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":5017600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":781688832,"data_stored":781533473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94340,"ondisk_log_size":94340,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482688,"kb_used_data":320496,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93889152,"statfs":{"total":96636764160,"available":96142491648,"internally_reserved":0,"allocated":328187904,"data_stored":327931000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241276,"kb_used_data":92972,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94130564,"statfs":{"total":96636764160,"available":96389697536,"internally_reserved":0,"allocated":95203328,"data_stored":94895378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520716,"kb_used_data":348860,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93851124,"statfs":{"total":96636764160,"available":96103550976,"internally_reserved":0,"allocated":357232640,"data_stored":356971659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,0,5,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357441536,"data_stored":357405406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96026624,"data_stored":95943525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328220672,"data_stored":328184542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:14.642 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:15.233 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:15.234 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:15.259 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-11T01:16:14.434807+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386085617,"num_objects":94270,"num_object_clones":0,"num_object_copies":188540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":94270,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":53405,"num_write_kb":213614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1226,"num_bytes_recovered":5017619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94343,"ondisk_log_size":94343,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1244680,"kb_used_data":762328,"kb_used_omap":12,"kb_used_meta":482291,"kb_avail":281870840,"statfs":{"total":289910292480,"available":288635740160,"internally_reserved":0,"allocated":780623872,"data_stored":779798037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":493865991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,10,1,8,16,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22261760,"num_objects":5435,"num_object_clones":0,"num_object_copies":10870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":5435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5435,"num_write_kb":21740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002715"},"pg_stats":[{"pgid":"2.f","version":"39'5840","reported_seq":5903,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.048517+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:10.048517+0000","last_peered":"2024-09-11T01:16:10.048517+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:10.048517+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5840,"log_dups_size":0,"ondisk_log_size":5840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23920640,"num_objects":5840,"num_object_clones":0,"num_object_copies":11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5840,"num_write_kb":23360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1717},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'5900","reported_seq":5970,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.839211+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:09.839211+0000","last_peered":"2024-09-11T01:16:09.839211+0000","last_clean":"2024-09-11T01:16:09.839211+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:09.839211+0000","last_undegraded":"2024-09-11T01:16:09.839211+0000","last_fullsized":"2024-09-11T01:16:09.839211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":5900,"log_dups_size":0,"ondisk_log_size":5900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142790,"num_objects":5896,"num_object_clones":0,"num_object_copies":11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5896,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3998,"num_write_kb":15992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5796","reported_seq":5868,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.815687+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:09.815687+0000","last_peered":"2024-09-11T01:16:09.815687+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:09.815687+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5796,"log_dups_size":0,"ondisk_log_size":5796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23724032,"num_objects":5792,"num_object_clones":0,"num_object_copies":11584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3857,"num_write_kb":15428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'5859","reported_seq":5894,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.827142+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:09.827142+0000","last_peered":"2024-09-11T01:16:09.827142+0000","last_clean":"2024-09-11T01:16:09.827142+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:09.827142+0000","last_undegraded":"2024-09-11T01:16:09.827142+0000","last_fullsized":"2024-09-11T01:16:09.827142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'5869","reported_seq":5980,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:10.082422+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:10.082422+0000","last_peered":"2024-09-11T01:16:10.082422+0000","last_clean":"2024-09-11T01:16:10.082422+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:10.082422+0000","last_undegraded":"2024-09-11T01:16:10.082422+0000","last_fullsized":"2024-09-11T01:16:10.082422+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":5869,"log_dups_size":0,"ondisk_log_size":5869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24011235,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2667,"num_write_kb":10668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'5892","reported_seq":6014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.092137+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:10.092137+0000","last_peered":"2024-09-11T01:16:10.092137+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:10.092137+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113635,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2380,"num_write_kb":9520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1773},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6044","reported_seq":6167,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.055221+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:10.055221+0000","last_peered":"2024-09-11T01:16:10.055221+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:10.055221+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24732614,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2797,"num_write_kb":11188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1757},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'5888","reported_seq":5997,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:10.108047+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:10.108047+0000","last_peered":"2024-09-11T01:16:10.108047+0000","last_clean":"2024-09-11T01:16:10.108047+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:10.108047+0000","last_undegraded":"2024-09-11T01:16:10.108047+0000","last_fullsized":"2024-09-11T01:16:10.108047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":5888,"log_dups_size":0,"ondisk_log_size":5888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24109056,"num_objects":5886,"num_object_clones":0,"num_object_copies":11772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2841,"num_write_kb":11364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'5857","reported_seq":8377,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:09.835817+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:09.835817+0000","last_peered":"2024-09-11T01:16:09.835817+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:09.835817+0000","last_undegraded":"2024-09-11T01:16:09.835817+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":5857,"log_dups_size":0,"ondisk_log_size":5857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23962083,"num_objects":5851,"num_object_clones":0,"num_object_copies":11702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":5851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3724,"num_write_kb":14896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1224,"num_bytes_recovered":5013504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2900},{"shards":"2","objects":2951}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'5943","reported_seq":6062,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.101214+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:10.101214+0000","last_peered":"2024-09-11T01:16:10.101214+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:10.101214+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24309760,"num_objects":5935,"num_object_clones":0,"num_object_copies":11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2307,"num_write_kb":9228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1830},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":87,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104336+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104336+0000","last_peered":"2024-09-11T01:15:20.104336+0000","last_clean":"2024-09-11T01:15:20.104336+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104336+0000","last_undegraded":"2024-09-11T01:15:20.104336+0000","last_fullsized":"2024-09-11T01:15:20.104336+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5788","reported_seq":5872,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.783530+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:09.783530+0000","last_peered":"2024-09-11T01:16:09.783530+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:09.783530+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5788,"log_dups_size":0,"ondisk_log_size":5788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23687651,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3557,"num_write_kb":14228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1757}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'5937","reported_seq":6020,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.803471+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:09.803471+0000","last_peered":"2024-09-11T01:16:09.803471+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:09.803471+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":5937,"log_dups_size":0,"ondisk_log_size":5937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24293376,"num_objects":5931,"num_object_clones":0,"num_object_copies":11862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3574,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'5927","reported_seq":6047,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.082699+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:10.082699+0000","last_peered":"2024-09-11T01:16:10.082699+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:10.082699+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":5927,"log_dups_size":0,"ondisk_log_size":5927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24268800,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2617,"num_write_kb":10468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1761},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'5970","reported_seq":6094,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.072569+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:10.072569+0000","last_peered":"2024-09-11T01:16:10.072569+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:10.072569+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2363,"num_write_kb":9452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'5964","reported_seq":6087,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.936716+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:09.936716+0000","last_peered":"2024-09-11T01:16:09.936716+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:09.936716+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":5964,"log_dups_size":0,"ondisk_log_size":5964,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1812},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'5866","reported_seq":5991,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.054902+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:10.054902+0000","last_peered":"2024-09-11T01:16:10.054902+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:10.054902+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":5866,"log_dups_size":0,"ondisk_log_size":5866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23978950,"num_objects":5856,"num_object_clones":0,"num_object_copies":11712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2523,"num_write_kb":10092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1702},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386085598,"num_objects":94268,"num_object_clones":0,"num_object_copies":188536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2900,"num_objects_unfound":0,"num_objects_dirty":94268,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":53403,"num_write_kb":213612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":5017600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":781688832,"data_stored":781533473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94340,"ondisk_log_size":94340,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482688,"kb_used_data":320496,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93889152,"statfs":{"total":96636764160,"available":96142491648,"internally_reserved":0,"allocated":328187904,"data_stored":327931000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":166063790},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241276,"kb_used_data":92972,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94130564,"statfs":{"total":96636764160,"available":96389697536,"internally_reserved":0,"allocated":95203328,"data_stored":94895378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520716,"kb_used_data":348860,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93851124,"statfs":{"total":96636764160,"available":96103550976,"internally_reserved":0,"allocated":357232640,"data_stored":356971659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,0,5,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357441536,"data_stored":357405406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96026624,"data_stored":95943525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":328220672,"data_stored":328184542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:15.261 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:15.261 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'5840", 'reported_seq': 5903, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.048517+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:10.048517+0000', 'last_peered': '2024-09-11T01:16:10.048517+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:10.048517+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 5840, 'log_dups_size': 0, 'ondisk_log_size': 5840, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23920640, 'num_objects': 5840, 'num_object_clones': 0, 'num_object_copies': 11680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5840, 'num_write_kb': 23360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1717}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.261 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:15.261 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'5796", 'reported_seq': 5868, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.815687+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:09.815687+0000', 'last_peered': '2024-09-11T01:16:09.815687+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:09.815687+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 5796, 'log_dups_size': 0, 'ondisk_log_size': 5796, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23724032, 'num_objects': 5792, 'num_object_clones': 0, 'num_object_copies': 11584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3857, 'num_write_kb': 15428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 1787}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:15.261 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:15.261 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'5892", 'reported_seq': 6014, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.092137+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:10.092137+0000', 'last_peered': '2024-09-11T01:16:10.092137+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:10.092137+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 5892, 'log_dups_size': 0, 'ondisk_log_size': 5892, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113635, 'num_objects': 5888, 'num_object_clones': 0, 'num_object_copies': 11776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5888, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2380, 'num_write_kb': 9520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1773}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'6044", 'reported_seq': 6167, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.055221+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:10.055221+0000', 'last_peered': '2024-09-11T01:16:10.055221+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:10.055221+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 6044, 'log_dups_size': 0, 'ondisk_log_size': 6044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24732614, 'num_objects': 6040, 'num_object_clones': 0, 'num_object_copies': 12080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6040, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 2797, 'num_write_kb': 11188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1757}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'5857", 'reported_seq': 8377, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:09.835817+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:09.835817+0000', 'last_peered': '2024-09-11T01:16:09.835817+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:09.835817+0000', 'last_undegraded': '2024-09-11T01:16:09.835817+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 5857, 'log_dups_size': 0, 'ondisk_log_size': 5857, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23962083, 'num_objects': 5851, 'num_object_clones': 0, 'num_object_copies': 11702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2900, 'num_objects_unfound': 0, 'num_objects_dirty': 5851, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3724, 'num_write_kb': 14896, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1224, 'num_bytes_recovered': 5013504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2900}, {'shards': '2', 'objects': 2951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'5943", 'reported_seq': 6062, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.101214+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:10.101214+0000', 'last_peered': '2024-09-11T01:16:10.101214+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:10.101214+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 5943, 'log_dups_size': 0, 'ondisk_log_size': 5943, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24309760, 'num_objects': 5935, 'num_object_clones': 0, 'num_object_copies': 11870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2307, 'num_write_kb': 9228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1830}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'5788", 'reported_seq': 5872, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.783530+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:09.783530+0000', 'last_peered': '2024-09-11T01:16:09.783530+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:09.783530+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 5788, 'log_dups_size': 0, 'ondisk_log_size': 5788, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23687651, 'num_objects': 5784, 'num_object_clones': 0, 'num_object_copies': 11568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5784, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3557, 'num_write_kb': 14228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 1757}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'5937", 'reported_seq': 6020, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.803471+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:09.803471+0000', 'last_peered': '2024-09-11T01:16:09.803471+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:09.803471+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 5937, 'log_dups_size': 0, 'ondisk_log_size': 5937, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24293376, 'num_objects': 5931, 'num_object_clones': 0, 'num_object_copies': 11862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3574, 'num_write_kb': 14296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 1754}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:15.262 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'5927", 'reported_seq': 6047, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.082699+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:10.082699+0000', 'last_peered': '2024-09-11T01:16:10.082699+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:10.082699+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 5927, 'log_dups_size': 0, 'ondisk_log_size': 5927, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24268800, 'num_objects': 5925, 'num_object_clones': 0, 'num_object_copies': 11850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2617, 'num_write_kb': 10468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1761}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'5970", 'reported_seq': 6094, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.072569+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:10.072569+0000', 'last_peered': '2024-09-11T01:16:10.072569+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:10.072569+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 5970, 'log_dups_size': 0, 'ondisk_log_size': 5970, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24420352, 'num_objects': 5962, 'num_object_clones': 0, 'num_object_copies': 11924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5962, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2363, 'num_write_kb': 9452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1770}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'5964", 'reported_seq': 6087, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.936716+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:09.936716+0000', 'last_peered': '2024-09-11T01:16:09.936716+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:09.936716+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 5964, 'log_dups_size': 0, 'ondisk_log_size': 5964, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24412160, 'num_objects': 5960, 'num_object_clones': 0, 'num_object_copies': 11920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2499, 'num_write_kb': 9996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1812}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:15.263 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'5866", 'reported_seq': 5991, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.054902+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:10.054902+0000', 'last_peered': '2024-09-11T01:16:10.054902+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:10.054902+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 5866, 'log_dups_size': 0, 'ondisk_log_size': 5866, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23978950, 'num_objects': 5856, 'num_object_clones': 0, 'num_object_copies': 11712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5856, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2523, 'num_write_kb': 10092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1702}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:15.263 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:15.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 99 20 97139 97119 3.83133 1.92969 0.02089 0.0203825 2024-09-11T01:16:16.003 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:16.003 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/188540 objects degraded (24.137%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":176,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94270,"data_bytes":386085617,"bytes_used":1274552320,"bytes_avail":288635740160,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":188540,"degraded_ratio":0.24137053145221174,"misplaced_objects":2900,"misplaced_total":188540,"misplaced_ratio":0.015381351437360772,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":90091,"recovering_keys_per_sec":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"write_bytes_sec":1854727,"read_op_per_sec":0,"write_op_per_sec":452},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:15:26.422427+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:16:16.034 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:16.034 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:16.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:16:16.329200+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0204803 2024-09-11T01:16:16.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:16:16.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 100 20 97653 97633 3.8131 2.00781 0.0381623 0.0204803 2024-09-11T01:16:16.603 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:16.603 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:16.632 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-11T01:16:16.435238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":397214449,"num_objects":96987,"num_object_clones":0,"num_object_copies":193974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2773,"num_objects_unfound":0,"num_objects_dirty":96987,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":56122,"num_write_kb":224482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":5537811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97060,"ondisk_log_size":97060,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1288888,"kb_used_data":776008,"kb_used_omap":12,"kb_used_meta":512819,"kb_avail":281826632,"statfs":{"total":289910292480,"available":288590471168,"internally_reserved":0,"allocated":794632192,"data_stored":793806357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":525126663},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,5,9,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33390592,"num_objects":8152,"num_object_clones":0,"num_object_copies":16304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-391,"num_objects_unfound":0,"num_objects_dirty":8152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8152,"num_write_kb":32608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":391,"num_bytes_recovered":1601536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002674"},"pg_stats":[{"pgid":"2.f","version":"39'5995","reported_seq":6058,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.069643+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:15.069643+0000","last_peered":"2024-09-11T01:16:15.069643+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:15.069643+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5995,"log_dups_size":0,"ondisk_log_size":5995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5995,"num_write_kb":23980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1872},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6055","reported_seq":6125,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:14.787516+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:14.787516+0000","last_peered":"2024-09-11T01:16:14.787516+0000","last_clean":"2024-09-11T01:16:14.787516+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:14.787516+0000","last_undegraded":"2024-09-11T01:16:14.787516+0000","last_fullsized":"2024-09-11T01:16:14.787516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6055,"log_dups_size":0,"ondisk_log_size":6055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24777670,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4153,"num_write_kb":16612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5983","reported_seq":6053,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.848106+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:14.848106+0000","last_peered":"2024-09-11T01:16:14.848106+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:14.848106+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5983,"log_dups_size":0,"ondisk_log_size":5983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24489984,"num_objects":5979,"num_object_clones":0,"num_object_copies":11958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":16176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1974}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6016","reported_seq":6051,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:14.797205+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:14.797205+0000","last_peered":"2024-09-11T01:16:14.797205+0000","last_clean":"2024-09-11T01:16:14.797205+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:14.797205+0000","last_undegraded":"2024-09-11T01:16:14.797205+0000","last_fullsized":"2024-09-11T01:16:14.797205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6016,"log_dups_size":0,"ondisk_log_size":6016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":6016,"num_object_clones":0,"num_object_copies":12032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6016,"num_write_kb":24064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6046","reported_seq":6157,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.077716+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:15.077716+0000","last_peered":"2024-09-11T01:16:15.077716+0000","last_clean":"2024-09-11T01:16:15.077716+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:15.077716+0000","last_undegraded":"2024-09-11T01:16:15.077716+0000","last_fullsized":"2024-09-11T01:16:15.077716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736227,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2844,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6069","reported_seq":6193,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.060809+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:15.060809+0000","last_peered":"2024-09-11T01:16:15.060809+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:15.060809+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838627,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1950},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6208","reported_seq":6329,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.122577+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:15.122577+0000","last_peered":"2024-09-11T01:16:15.122577+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:15.122577+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25404358,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2961,"num_write_kb":11844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6063","reported_seq":6172,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.066763+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:15.066763+0000","last_peered":"2024-09-11T01:16:15.066763+0000","last_clean":"2024-09-11T01:16:15.066763+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:15.066763+0000","last_undegraded":"2024-09-11T01:16:15.066763+0000","last_fullsized":"2024-09-11T01:16:15.066763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24825856,"num_objects":6061,"num_object_clones":0,"num_object_copies":12122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3016,"num_write_kb":12064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6044","reported_seq":8819,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:14.863363+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:14.863363+0000","last_peered":"2024-09-11T01:16:14.863363+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:14.863363+0000","last_undegraded":"2024-09-11T01:16:14.863363+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24728035,"num_objects":6038,"num_object_clones":0,"num_object_copies":12076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2773,"num_objects_unfound":0,"num_objects_dirty":6038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3911,"num_write_kb":15644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":5533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2773},{"shards":"2","objects":3265}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6114","reported_seq":6232,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.108081+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:15.108081+0000","last_peered":"2024-09-11T01:16:15.108081+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:15.108081+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6114,"log_dups_size":0,"ondisk_log_size":6114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25010176,"num_objects":6106,"num_object_clones":0,"num_object_copies":12212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2478,"num_write_kb":9912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2001},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5972","reported_seq":6056,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.823638+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:14.823638+0000","last_peered":"2024-09-11T01:16:14.823638+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:14.823638+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24441315,"num_objects":5968,"num_object_clones":0,"num_object_copies":11936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3741,"num_write_kb":14964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6115","reported_seq":6198,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.848576+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:14.848576+0000","last_peered":"2024-09-11T01:16:14.848576+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:14.848576+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6115,"log_dups_size":0,"ondisk_log_size":6115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3752,"num_write_kb":15008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6091","reported_seq":6210,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.108047+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:15.108047+0000","last_peered":"2024-09-11T01:16:15.108047+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:15.108047+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2781,"num_write_kb":11124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1925},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6137","reported_seq":6260,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.111038+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:15.111038+0000","last_peered":"2024-09-11T01:16:15.111038+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:15.111038+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6137,"log_dups_size":0,"ondisk_log_size":6137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2530,"num_write_kb":10120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6121","reported_seq":6244,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.052589+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:15.052589+0000","last_peered":"2024-09-11T01:16:15.052589+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:15.052589+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6121,"log_dups_size":0,"ondisk_log_size":6121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2656,"num_write_kb":10624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1969},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6028","reported_seq":6153,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.083101+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:15.083101+0000","last_peered":"2024-09-11T01:16:15.083101+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:15.083101+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6028,"log_dups_size":0,"ondisk_log_size":6028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24642502,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1864},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":397214430,"num_objects":96985,"num_object_clones":0,"num_object_copies":193970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2773,"num_objects_unfound":0,"num_objects_dirty":96985,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":56120,"num_write_kb":224480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1352,"num_bytes_recovered":5537792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":795553792,"data_stored":795398433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97057,"ondisk_log_size":97057,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510580,"kb_used_data":326180,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93861260,"statfs":{"total":96636764160,"available":96113930240,"internally_reserved":0,"allocated":334008320,"data_stored":333751416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789964,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241276,"kb_used_data":92972,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94130564,"statfs":{"total":96636764160,"available":96389697536,"internally_reserved":0,"allocated":95203328,"data_stored":94895378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537032,"kb_used_data":356856,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93834808,"statfs":{"total":96636764160,"available":96086843392,"internally_reserved":0,"allocated":365420544,"data_stored":365159563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365547520,"data_stored":365511390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96026624,"data_stored":95943525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333979648,"data_stored":333943518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:17.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 101 20 98177 98157 3.79561 2.04688 0.0273674 0.0205733 2024-09-11T01:16:18.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 102 20 98736 98716 3.77981 2.18359 0.0215834 0.0206615 2024-09-11T01:16:19.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 103 20 99298 99278 3.76442 2.19531 0.0477681 0.0207459 2024-09-11T01:16:19.634 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:20.227 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:20.230 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:20.256 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-11T01:16:18.435727+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":397214449,"num_objects":96987,"num_object_clones":0,"num_object_copies":193974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2773,"num_objects_unfound":0,"num_objects_dirty":96987,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":56122,"num_write_kb":224482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1353,"num_bytes_recovered":5537811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97060,"ondisk_log_size":97060,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1289080,"kb_used_data":784516,"kb_used_omap":12,"kb_used_meta":504499,"kb_avail":281826440,"statfs":{"total":289910292480,"available":288590274560,"internally_reserved":0,"allocated":803344384,"data_stored":802518549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":516606983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,7,8,17],"upper_bound":128},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22446080,"num_objects":5480,"num_object_clones":0,"num_object_copies":10960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-264,"num_objects_unfound":0,"num_objects_dirty":5480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5480,"num_write_kb":21920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":1081344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002755"},"pg_stats":[{"pgid":"2.f","version":"39'5995","reported_seq":6058,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.069643+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:15.069643+0000","last_peered":"2024-09-11T01:16:15.069643+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:15.069643+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":5995,"log_dups_size":0,"ondisk_log_size":5995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24555520,"num_objects":5995,"num_object_clones":0,"num_object_copies":11990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5995,"num_write_kb":23980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1872},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6055","reported_seq":6125,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:14.787516+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:14.787516+0000","last_peered":"2024-09-11T01:16:14.787516+0000","last_clean":"2024-09-11T01:16:14.787516+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:14.787516+0000","last_undegraded":"2024-09-11T01:16:14.787516+0000","last_fullsized":"2024-09-11T01:16:14.787516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6055,"log_dups_size":0,"ondisk_log_size":6055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24777670,"num_objects":6051,"num_object_clones":0,"num_object_copies":12102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6051,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4153,"num_write_kb":16612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'5983","reported_seq":6053,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.848106+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:14.848106+0000","last_peered":"2024-09-11T01:16:14.848106+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:14.848106+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":5983,"log_dups_size":0,"ondisk_log_size":5983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24489984,"num_objects":5979,"num_object_clones":0,"num_object_copies":11958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":16176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":1974}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6016","reported_seq":6051,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:14.797205+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:14.797205+0000","last_peered":"2024-09-11T01:16:14.797205+0000","last_clean":"2024-09-11T01:16:14.797205+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:14.797205+0000","last_undegraded":"2024-09-11T01:16:14.797205+0000","last_fullsized":"2024-09-11T01:16:14.797205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6016,"log_dups_size":0,"ondisk_log_size":6016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":6016,"num_object_clones":0,"num_object_copies":12032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6016,"num_write_kb":24064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6046","reported_seq":6157,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.077716+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:15.077716+0000","last_peered":"2024-09-11T01:16:15.077716+0000","last_clean":"2024-09-11T01:16:15.077716+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:15.077716+0000","last_undegraded":"2024-09-11T01:16:15.077716+0000","last_fullsized":"2024-09-11T01:16:15.077716+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6046,"log_dups_size":0,"ondisk_log_size":6046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736227,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2844,"num_write_kb":11376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6069","reported_seq":6193,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.060809+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:15.060809+0000","last_peered":"2024-09-11T01:16:15.060809+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:15.060809+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6069,"log_dups_size":0,"ondisk_log_size":6069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838627,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2557,"num_write_kb":10228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1950},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6208","reported_seq":6329,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.122577+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:15.122577+0000","last_peered":"2024-09-11T01:16:15.122577+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:15.122577+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6208,"log_dups_size":0,"ondisk_log_size":6208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25404358,"num_objects":6204,"num_object_clones":0,"num_object_copies":12408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6204,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2961,"num_write_kb":11844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1921},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6063","reported_seq":6172,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.066763+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:15.066763+0000","last_peered":"2024-09-11T01:16:15.066763+0000","last_clean":"2024-09-11T01:16:15.066763+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:15.066763+0000","last_undegraded":"2024-09-11T01:16:15.066763+0000","last_fullsized":"2024-09-11T01:16:15.066763+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24825856,"num_objects":6061,"num_object_clones":0,"num_object_copies":12122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3016,"num_write_kb":12064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6044","reported_seq":8819,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:14.863363+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:14.863363+0000","last_peered":"2024-09-11T01:16:14.863363+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:14.863363+0000","last_undegraded":"2024-09-11T01:16:14.863363+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24728035,"num_objects":6038,"num_object_clones":0,"num_object_copies":12076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2773,"num_objects_unfound":0,"num_objects_dirty":6038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3911,"num_write_kb":15644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1351,"num_bytes_recovered":5533696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2773},{"shards":"2","objects":3265}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6114","reported_seq":6232,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.108081+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:15.108081+0000","last_peered":"2024-09-11T01:16:15.108081+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:15.108081+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6114,"log_dups_size":0,"ondisk_log_size":6114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25010176,"num_objects":6106,"num_object_clones":0,"num_object_copies":12212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2478,"num_write_kb":9912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2001},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'5972","reported_seq":6056,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.823638+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:14.823638+0000","last_peered":"2024-09-11T01:16:14.823638+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:14.823638+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24441315,"num_objects":5968,"num_object_clones":0,"num_object_copies":11936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3741,"num_write_kb":14964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":1941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6115","reported_seq":6198,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.848576+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:14.848576+0000","last_peered":"2024-09-11T01:16:14.848576+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:14.848576+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6115,"log_dups_size":0,"ondisk_log_size":6115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25022464,"num_objects":6109,"num_object_clones":0,"num_object_copies":12218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3752,"num_write_kb":15008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":1932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6091","reported_seq":6210,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.108047+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:15.108047+0000","last_peered":"2024-09-11T01:16:15.108047+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:15.108047+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6091,"log_dups_size":0,"ondisk_log_size":6091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2781,"num_write_kb":11124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1925},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6137","reported_seq":6260,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.111038+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:15.111038+0000","last_peered":"2024-09-11T01:16:15.111038+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:15.111038+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6137,"log_dups_size":0,"ondisk_log_size":6137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25104384,"num_objects":6129,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2530,"num_write_kb":10120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6121","reported_seq":6244,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.052589+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:15.052589+0000","last_peered":"2024-09-11T01:16:15.052589+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:15.052589+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6121,"log_dups_size":0,"ondisk_log_size":6121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25055232,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2656,"num_write_kb":10624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1969},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6028","reported_seq":6153,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.083101+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:15.083101+0000","last_peered":"2024-09-11T01:16:15.083101+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:15.083101+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6028,"log_dups_size":0,"ondisk_log_size":6028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24642502,"num_objects":6018,"num_object_clones":0,"num_object_copies":12036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2685,"num_write_kb":10740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1864},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":397214430,"num_objects":96985,"num_object_clones":0,"num_object_copies":193970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2773,"num_objects_unfound":0,"num_objects_dirty":96985,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":56120,"num_write_kb":224480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1352,"num_bytes_recovered":5537792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":804409344,"data_stored":804253985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97057,"ondisk_log_size":97057,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510580,"kb_used_data":326180,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93861260,"statfs":{"total":96636764160,"available":96113930240,"internally_reserved":0,"allocated":334008320,"data_stored":333751416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241468,"kb_used_data":101480,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94130372,"statfs":{"total":96636764160,"available":96389500928,"internally_reserved":0,"allocated":103915520,"data_stored":103607570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537032,"kb_used_data":356856,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93834808,"statfs":{"total":96636764160,"available":96086843392,"internally_reserved":0,"allocated":365420544,"data_stored":365159563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365547520,"data_stored":365511390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104882176,"data_stored":104799077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333979648,"data_stored":333943518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:20.257 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:20.330 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 104 20 99805 99785 3.74727 1.98047 0.0218735 0.0208403 2024-09-11T01:16:20.835 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:20.835 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:20.864 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-11T01:16:20.436170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408216305,"num_objects":99673,"num_object_clones":0,"num_object_copies":199346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99673,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58808,"num_write_kb":235226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":6107155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99746,"ondisk_log_size":99746,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294068,"kb_used_data":797824,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281821452,"statfs":{"total":289910292480,"available":288585166848,"internally_reserved":0,"allocated":816971776,"data_stored":816145941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":508087303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,5,11,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-403,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8166,"num_write_kb":32664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002737"},"pg_stats":[{"pgid":"2.f","version":"39'6156","reported_seq":6219,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.058912+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:20.058912+0000","last_peered":"2024-09-11T01:16:20.058912+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:20.058912+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2033},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6217","reported_seq":6287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.820170+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:19.820170+0000","last_peered":"2024-09-11T01:16:19.820170+0000","last_clean":"2024-09-11T01:16:19.820170+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:19.820170+0000","last_undegraded":"2024-09-11T01:16:19.820170+0000","last_fullsized":"2024-09-11T01:16:19.820170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25441222,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4315,"num_write_kb":17260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6158","reported_seq":6231,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.822451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:19.822451+0000","last_peered":"2024-09-11T01:16:19.822451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:19.822451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6158,"log_dups_size":0,"ondisk_log_size":6158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25206784,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6188","reported_seq":6223,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.790088+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:19.790088+0000","last_peered":"2024-09-11T01:16:19.790088+0000","last_clean":"2024-09-11T01:16:19.790088+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:19.790088+0000","last_undegraded":"2024-09-11T01:16:19.790088+0000","last_fullsized":"2024-09-11T01:16:19.790088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6188,"log_dups_size":0,"ondisk_log_size":6188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6188,"num_write_kb":24752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6205","reported_seq":6316,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.142560+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:20.142560+0000","last_peered":"2024-09-11T01:16:20.142560+0000","last_clean":"2024-09-11T01:16:20.142560+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:20.142560+0000","last_undegraded":"2024-09-11T01:16:20.142560+0000","last_fullsized":"2024-09-11T01:16:20.142560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387491,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6243","reported_seq":6367,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.134618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:20.134618+0000","last_peered":"2024-09-11T01:16:20.134618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:20.134618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25551331,"num_objects":6239,"num_object_clones":0,"num_object_copies":12478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2731,"num_write_kb":10924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6377","reported_seq":6500,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.081018+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:20.081018+0000","last_peered":"2024-09-11T01:16:20.081018+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:20.081018+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6377,"log_dups_size":0,"ondisk_log_size":6377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26096582,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6225","reported_seq":6332,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.125171+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:20.125171+0000","last_peered":"2024-09-11T01:16:20.125171+0000","last_clean":"2024-09-11T01:16:20.125171+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:20.125171+0000","last_undegraded":"2024-09-11T01:16:20.125171+0000","last_fullsized":"2024-09-11T01:16:20.125171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6225,"log_dups_size":0,"ondisk_log_size":6225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3178,"num_write_kb":12712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6218","reported_seq":9271,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:19.872584+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:19.872584+0000","last_peered":"2024-09-11T01:16:19.872584+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:19.872584+0000","last_undegraded":"2024-09-11T01:16:19.872584+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6218,"log_dups_size":0,"ondisk_log_size":6218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25440739,"num_objects":6212,"num_object_clones":0,"num_object_copies":12424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":6212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4085,"num_write_kb":16340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1490,"num_bytes_recovered":6103040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2634},{"shards":"2","objects":3578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6281","reported_seq":6400,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.065661+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:20.065661+0000","last_peered":"2024-09-11T01:16:20.065661+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:20.065661+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2168},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6131","reported_seq":6213,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.800258+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:19.800258+0000","last_peered":"2024-09-11T01:16:19.800258+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:19.800258+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6131,"log_dups_size":0,"ondisk_log_size":6131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3900,"num_write_kb":15600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6283","reported_seq":6364,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.840342+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:19.840342+0000","last_peered":"2024-09-11T01:16:19.840342+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:19.840342+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6265","reported_seq":6385,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.133670+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:20.133670+0000","last_peered":"2024-09-11T01:16:20.133670+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:20.133670+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6265,"log_dups_size":0,"ondisk_log_size":6265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2099},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6316","reported_seq":6440,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.090714+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:20.090714+0000","last_peered":"2024-09-11T01:16:20.090714+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:20.090714+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2709,"num_write_kb":10836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6289","reported_seq":6408,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.098560+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:20.098560+0000","last_peered":"2024-09-11T01:16:20.098560+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:20.098560+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6289,"log_dups_size":0,"ondisk_log_size":6289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6191","reported_seq":6316,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.038439+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:20.038439+0000","last_peered":"2024-09-11T01:16:20.038439+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:20.038439+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25310150,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2027},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408216286,"num_objects":99671,"num_object_clones":0,"num_object_copies":199342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99671,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58806,"num_write_kb":235224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":6107136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":818057216,"data_stored":817901857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99743,"ondisk_log_size":99743,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503400,"kb_used_data":331480,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93868440,"statfs":{"total":96636764160,"available":96121282560,"internally_reserved":0,"allocated":339435520,"data_stored":339178616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241468,"kb_used_data":101480,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94130372,"statfs":{"total":96636764160,"available":96389500928,"internally_reserved":0,"allocated":103915520,"data_stored":103607570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549200,"kb_used_data":364864,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93822640,"statfs":{"total":96636764160,"available":96074383360,"internally_reserved":0,"allocated":373620736,"data_stored":373359755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373776384,"data_stored":373740254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104882176,"data_stored":104799077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339398656,"data_stored":339362526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'6156", 'reported_seq': 6219, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.058912+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:20.058912+0000', 'last_peered': '2024-09-11T01:16:20.058912+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:20.058912+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 6156, 'log_dups_size': 0, 'ondisk_log_size': 6156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25214976, 'num_objects': 6156, 'num_object_clones': 0, 'num_object_copies': 12312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6156, 'num_write_kb': 24624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2033}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'6158", 'reported_seq': 6231, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:19.822451+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:19.822451+0000', 'last_peered': '2024-09-11T01:16:19.822451+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:19.822451+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 6158, 'log_dups_size': 0, 'ondisk_log_size': 6158, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25206784, 'num_objects': 6154, 'num_object_clones': 0, 'num_object_copies': 12308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4219, 'num_write_kb': 16876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 2149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'6243", 'reported_seq': 6367, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.134618+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:20.134618+0000', 'last_peered': '2024-09-11T01:16:20.134618+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:20.134618+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 6243, 'log_dups_size': 0, 'ondisk_log_size': 6243, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25551331, 'num_objects': 6239, 'num_object_clones': 0, 'num_object_copies': 12478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6239, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2731, 'num_write_kb': 10924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2124}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.866 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'6377", 'reported_seq': 6500, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.081018+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:20.081018+0000', 'last_peered': '2024-09-11T01:16:20.081018+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:20.081018+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 6377, 'log_dups_size': 0, 'ondisk_log_size': 6377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26096582, 'num_objects': 6373, 'num_object_clones': 0, 'num_object_copies': 12746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6373, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3130, 'num_write_kb': 12520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2090}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'6218", 'reported_seq': 9271, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:19.872584+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:19.872584+0000', 'last_peered': '2024-09-11T01:16:19.872584+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:19.872584+0000', 'last_undegraded': '2024-09-11T01:16:19.872584+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 6218, 'log_dups_size': 0, 'ondisk_log_size': 6218, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25440739, 'num_objects': 6212, 'num_object_clones': 0, 'num_object_copies': 12424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2634, 'num_objects_unfound': 0, 'num_objects_dirty': 6212, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4085, 'num_write_kb': 16340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1490, 'num_bytes_recovered': 6103040, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2634}, {'shards': '2', 'objects': 3578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'6281", 'reported_seq': 6400, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.065661+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:20.065661+0000', 'last_peered': '2024-09-11T01:16:20.065661+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:20.065661+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 6281, 'log_dups_size': 0, 'ondisk_log_size': 6281, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25694208, 'num_objects': 6273, 'num_object_clones': 0, 'num_object_copies': 12546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2645, 'num_write_kb': 10580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2168}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'6131", 'reported_seq': 6213, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:19.800258+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:19.800258+0000', 'last_peered': '2024-09-11T01:16:19.800258+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:19.800258+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 6131, 'log_dups_size': 0, 'ondisk_log_size': 6131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25092579, 'num_objects': 6127, 'num_object_clones': 0, 'num_object_copies': 12254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6127, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3900, 'num_write_kb': 15600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 2100}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'6283", 'reported_seq': 6364, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:19.840342+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:19.840342+0000', 'last_peered': '2024-09-11T01:16:19.840342+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:19.840342+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 6283, 'log_dups_size': 0, 'ondisk_log_size': 6283, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25710592, 'num_objects': 6277, 'num_object_clones': 0, 'num_object_copies': 12554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3920, 'num_write_kb': 15680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 2100}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:20.867 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'6265", 'reported_seq': 6385, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.133670+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:20.133670+0000', 'last_peered': '2024-09-11T01:16:20.133670+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:20.133670+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 6265, 'log_dups_size': 0, 'ondisk_log_size': 6265, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25653248, 'num_objects': 6263, 'num_object_clones': 0, 'num_object_copies': 12526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6263, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2955, 'num_write_kb': 11820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2099}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:20.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'6316", 'reported_seq': 6440, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.090714+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:20.090714+0000', 'last_peered': '2024-09-11T01:16:20.090714+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:20.090714+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 6316, 'log_dups_size': 0, 'ondisk_log_size': 6316, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25837568, 'num_objects': 6308, 'num_object_clones': 0, 'num_object_copies': 12616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2709, 'num_write_kb': 10836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2116}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:20.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'6289", 'reported_seq': 6408, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.098560+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:20.098560+0000', 'last_peered': '2024-09-11T01:16:20.098560+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:20.098560+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 6289, 'log_dups_size': 0, 'ondisk_log_size': 6289, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25743360, 'num_objects': 6285, 'num_object_clones': 0, 'num_object_copies': 12570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2824, 'num_write_kb': 11296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2137}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:20.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'6191", 'reported_seq': 6316, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.038439+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:20.038439+0000', 'last_peered': '2024-09-11T01:16:20.038439+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:20.038439+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 6191, 'log_dups_size': 0, 'ondisk_log_size': 6191, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25310150, 'num_objects': 6181, 'num_object_clones': 0, 'num_object_copies': 12362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6181, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2848, 'num_write_kb': 11392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2027}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:20.868 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:21.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 105 20 100302 100282 3.73001 1.94141 0.0649668 0.0209362 2024-09-11T01:16:21.574 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:21.574 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/199346 objects degraded (22.829%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":182,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99673,"data_bytes":408216305,"bytes_used":1325125632,"bytes_avail":288585166848,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":199346,"degraded_ratio":0.22828649684468211,"misplaced_objects":2634,"misplaced_total":199346,"misplaced_ratio":0.013213207187503136,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":137525,"recovering_keys_per_sec":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"write_bytes_sec":2786692,"read_op_per_sec":0,"write_op_per_sec":680},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:21.605 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:21.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:22.199 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:22.202 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:22.231 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-11T01:16:20.436170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408216305,"num_objects":99673,"num_object_clones":0,"num_object_copies":199346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99673,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58808,"num_write_kb":235226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":6107155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99746,"ondisk_log_size":99746,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1294068,"kb_used_data":797824,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281821452,"statfs":{"total":289910292480,"available":288585166848,"internally_reserved":0,"allocated":816971776,"data_stored":816145941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":508087303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,5,11,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-403,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8166,"num_write_kb":32664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002737"},"pg_stats":[{"pgid":"2.f","version":"39'6156","reported_seq":6219,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.058912+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:20.058912+0000","last_peered":"2024-09-11T01:16:20.058912+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:20.058912+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2033},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6217","reported_seq":6287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.820170+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:19.820170+0000","last_peered":"2024-09-11T01:16:19.820170+0000","last_clean":"2024-09-11T01:16:19.820170+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:19.820170+0000","last_undegraded":"2024-09-11T01:16:19.820170+0000","last_fullsized":"2024-09-11T01:16:19.820170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25441222,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4315,"num_write_kb":17260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6158","reported_seq":6231,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.822451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:19.822451+0000","last_peered":"2024-09-11T01:16:19.822451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:19.822451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6158,"log_dups_size":0,"ondisk_log_size":6158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25206784,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6188","reported_seq":6223,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.790088+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:19.790088+0000","last_peered":"2024-09-11T01:16:19.790088+0000","last_clean":"2024-09-11T01:16:19.790088+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:19.790088+0000","last_undegraded":"2024-09-11T01:16:19.790088+0000","last_fullsized":"2024-09-11T01:16:19.790088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6188,"log_dups_size":0,"ondisk_log_size":6188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6188,"num_write_kb":24752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6205","reported_seq":6316,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.142560+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:20.142560+0000","last_peered":"2024-09-11T01:16:20.142560+0000","last_clean":"2024-09-11T01:16:20.142560+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:20.142560+0000","last_undegraded":"2024-09-11T01:16:20.142560+0000","last_fullsized":"2024-09-11T01:16:20.142560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387491,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6243","reported_seq":6367,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.134618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:20.134618+0000","last_peered":"2024-09-11T01:16:20.134618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:20.134618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25551331,"num_objects":6239,"num_object_clones":0,"num_object_copies":12478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2731,"num_write_kb":10924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6377","reported_seq":6500,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.081018+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:20.081018+0000","last_peered":"2024-09-11T01:16:20.081018+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:20.081018+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6377,"log_dups_size":0,"ondisk_log_size":6377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26096582,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6225","reported_seq":6332,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.125171+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:20.125171+0000","last_peered":"2024-09-11T01:16:20.125171+0000","last_clean":"2024-09-11T01:16:20.125171+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:20.125171+0000","last_undegraded":"2024-09-11T01:16:20.125171+0000","last_fullsized":"2024-09-11T01:16:20.125171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6225,"log_dups_size":0,"ondisk_log_size":6225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3178,"num_write_kb":12712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6218","reported_seq":9271,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:19.872584+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:19.872584+0000","last_peered":"2024-09-11T01:16:19.872584+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:19.872584+0000","last_undegraded":"2024-09-11T01:16:19.872584+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6218,"log_dups_size":0,"ondisk_log_size":6218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25440739,"num_objects":6212,"num_object_clones":0,"num_object_copies":12424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":6212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4085,"num_write_kb":16340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1490,"num_bytes_recovered":6103040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2634},{"shards":"2","objects":3578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6281","reported_seq":6400,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.065661+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:20.065661+0000","last_peered":"2024-09-11T01:16:20.065661+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:20.065661+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2168},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6131","reported_seq":6213,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.800258+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:19.800258+0000","last_peered":"2024-09-11T01:16:19.800258+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:19.800258+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6131,"log_dups_size":0,"ondisk_log_size":6131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3900,"num_write_kb":15600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6283","reported_seq":6364,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.840342+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:19.840342+0000","last_peered":"2024-09-11T01:16:19.840342+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:19.840342+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6265","reported_seq":6385,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.133670+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:20.133670+0000","last_peered":"2024-09-11T01:16:20.133670+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:20.133670+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6265,"log_dups_size":0,"ondisk_log_size":6265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2099},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6316","reported_seq":6440,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.090714+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:20.090714+0000","last_peered":"2024-09-11T01:16:20.090714+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:20.090714+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2709,"num_write_kb":10836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6289","reported_seq":6408,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.098560+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:20.098560+0000","last_peered":"2024-09-11T01:16:20.098560+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:20.098560+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6289,"log_dups_size":0,"ondisk_log_size":6289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6191","reported_seq":6316,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.038439+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:20.038439+0000","last_peered":"2024-09-11T01:16:20.038439+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:20.038439+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25310150,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2027},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408216286,"num_objects":99671,"num_object_clones":0,"num_object_copies":199342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99671,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58806,"num_write_kb":235224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":6107136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":818057216,"data_stored":817901857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99743,"ondisk_log_size":99743,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503400,"kb_used_data":331480,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93868440,"statfs":{"total":96636764160,"available":96121282560,"internally_reserved":0,"allocated":339435520,"data_stored":339178616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241468,"kb_used_data":101480,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94130372,"statfs":{"total":96636764160,"available":96389500928,"internally_reserved":0,"allocated":103915520,"data_stored":103607570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549200,"kb_used_data":364864,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93822640,"statfs":{"total":96636764160,"available":96074383360,"internally_reserved":0,"allocated":373620736,"data_stored":373359755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373776384,"data_stored":373740254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104882176,"data_stored":104799077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339398656,"data_stored":339362526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:22.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 106 20 100805 100785 3.71335 1.96484 0.0230231 0.021031 2024-09-11T01:16:23.332 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 107 20 101369 101349 3.69924 2.20312 0.0151026 0.0211114 2024-09-11T01:16:24.336 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 108 20 101922 101902 3.68487 2.16016 0.0195855 0.0211939 2024-09-11T01:16:25.233 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:25.336 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 109 20 102450 102430 3.66998 2.0625 0.0849515 0.0212787 2024-09-11T01:16:25.803 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:25.803 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:25.826 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-11T01:16:24.437147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408216305,"num_objects":99673,"num_object_clones":0,"num_object_copies":199346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99673,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58808,"num_write_kb":235226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":6107155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99746,"ondisk_log_size":99746,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1306596,"kb_used_data":806196,"kb_used_omap":12,"kb_used_meta":500339,"kb_avail":281808924,"statfs":{"total":289910292480,"available":288572338176,"internally_reserved":0,"allocated":825544704,"data_stored":824718869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":512347143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,5,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22130688,"num_objects":5403,"num_object_clones":0,"num_object_copies":10806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":5403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002741"},"pg_stats":[{"pgid":"2.f","version":"39'6156","reported_seq":6219,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.058912+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:20.058912+0000","last_peered":"2024-09-11T01:16:20.058912+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:20.058912+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2033},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6217","reported_seq":6287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.820170+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:19.820170+0000","last_peered":"2024-09-11T01:16:19.820170+0000","last_clean":"2024-09-11T01:16:19.820170+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:19.820170+0000","last_undegraded":"2024-09-11T01:16:19.820170+0000","last_fullsized":"2024-09-11T01:16:19.820170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25441222,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4315,"num_write_kb":17260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6158","reported_seq":6231,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.822451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:19.822451+0000","last_peered":"2024-09-11T01:16:19.822451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:19.822451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6158,"log_dups_size":0,"ondisk_log_size":6158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25206784,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6188","reported_seq":6223,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.790088+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:19.790088+0000","last_peered":"2024-09-11T01:16:19.790088+0000","last_clean":"2024-09-11T01:16:19.790088+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:19.790088+0000","last_undegraded":"2024-09-11T01:16:19.790088+0000","last_fullsized":"2024-09-11T01:16:19.790088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6188,"log_dups_size":0,"ondisk_log_size":6188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6188,"num_write_kb":24752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6205","reported_seq":6316,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.142560+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:20.142560+0000","last_peered":"2024-09-11T01:16:20.142560+0000","last_clean":"2024-09-11T01:16:20.142560+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:20.142560+0000","last_undegraded":"2024-09-11T01:16:20.142560+0000","last_fullsized":"2024-09-11T01:16:20.142560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387491,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6243","reported_seq":6367,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.134618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:20.134618+0000","last_peered":"2024-09-11T01:16:20.134618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:20.134618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25551331,"num_objects":6239,"num_object_clones":0,"num_object_copies":12478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2731,"num_write_kb":10924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6377","reported_seq":6500,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.081018+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:20.081018+0000","last_peered":"2024-09-11T01:16:20.081018+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:20.081018+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6377,"log_dups_size":0,"ondisk_log_size":6377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26096582,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6225","reported_seq":6332,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.125171+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:20.125171+0000","last_peered":"2024-09-11T01:16:20.125171+0000","last_clean":"2024-09-11T01:16:20.125171+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:20.125171+0000","last_undegraded":"2024-09-11T01:16:20.125171+0000","last_fullsized":"2024-09-11T01:16:20.125171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6225,"log_dups_size":0,"ondisk_log_size":6225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3178,"num_write_kb":12712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6218","reported_seq":9271,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:19.872584+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:19.872584+0000","last_peered":"2024-09-11T01:16:19.872584+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:19.872584+0000","last_undegraded":"2024-09-11T01:16:19.872584+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6218,"log_dups_size":0,"ondisk_log_size":6218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25440739,"num_objects":6212,"num_object_clones":0,"num_object_copies":12424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":6212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4085,"num_write_kb":16340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1490,"num_bytes_recovered":6103040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2634},{"shards":"2","objects":3578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6281","reported_seq":6400,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.065661+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:20.065661+0000","last_peered":"2024-09-11T01:16:20.065661+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:20.065661+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2168},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6131","reported_seq":6213,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.800258+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:19.800258+0000","last_peered":"2024-09-11T01:16:19.800258+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:19.800258+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6131,"log_dups_size":0,"ondisk_log_size":6131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3900,"num_write_kb":15600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6283","reported_seq":6364,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.840342+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:19.840342+0000","last_peered":"2024-09-11T01:16:19.840342+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:19.840342+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6265","reported_seq":6385,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.133670+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:20.133670+0000","last_peered":"2024-09-11T01:16:20.133670+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:20.133670+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6265,"log_dups_size":0,"ondisk_log_size":6265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2099},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6316","reported_seq":6440,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.090714+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:20.090714+0000","last_peered":"2024-09-11T01:16:20.090714+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:20.090714+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2709,"num_write_kb":10836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6289","reported_seq":6408,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.098560+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:20.098560+0000","last_peered":"2024-09-11T01:16:20.098560+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:20.098560+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6289,"log_dups_size":0,"ondisk_log_size":6289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6191","reported_seq":6316,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.038439+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:20.038439+0000","last_peered":"2024-09-11T01:16:20.038439+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:20.038439+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25310150,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2027},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408216286,"num_objects":99671,"num_object_clones":0,"num_object_copies":199342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99671,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58806,"num_write_kb":235224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":6107136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":826753024,"data_stored":826597665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99743,"ondisk_log_size":99743,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503400,"kb_used_data":331480,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93868440,"statfs":{"total":96636764160,"available":96121282560,"internally_reserved":0,"allocated":339435520,"data_stored":339178616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789966,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253996,"kb_used_data":109852,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94117844,"statfs":{"total":96636764160,"available":96376672256,"internally_reserved":0,"allocated":112488448,"data_stored":112180498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549200,"kb_used_data":364864,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93822640,"statfs":{"total":96636764160,"available":96074383360,"internally_reserved":0,"allocated":373620736,"data_stored":373359755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373776384,"data_stored":373740254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113577984,"data_stored":113494885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339398656,"data_stored":339362526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:25.827 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:26.337 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 110 20 102928 102908 3.65355 1.86719 0.00849667 0.0213746 2024-09-11T01:16:26.433 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:26.434 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:26.461 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-11T01:16:24.437147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408216305,"num_objects":99673,"num_object_clones":0,"num_object_copies":199346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99673,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58808,"num_write_kb":235226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1492,"num_bytes_recovered":6107155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99746,"ondisk_log_size":99746,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1306596,"kb_used_data":806196,"kb_used_omap":12,"kb_used_meta":500339,"kb_avail":281808924,"statfs":{"total":289910292480,"available":288572338176,"internally_reserved":0,"allocated":825544704,"data_stored":824718869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":512347143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,5,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22130688,"num_objects":5403,"num_object_clones":0,"num_object_copies":10806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":5403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002741"},"pg_stats":[{"pgid":"2.f","version":"39'6156","reported_seq":6219,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.058912+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:20.058912+0000","last_peered":"2024-09-11T01:16:20.058912+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:20.058912+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25214976,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6156,"num_write_kb":24624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2033},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6217","reported_seq":6287,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.820170+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:19.820170+0000","last_peered":"2024-09-11T01:16:19.820170+0000","last_clean":"2024-09-11T01:16:19.820170+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:19.820170+0000","last_undegraded":"2024-09-11T01:16:19.820170+0000","last_fullsized":"2024-09-11T01:16:19.820170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6217,"log_dups_size":0,"ondisk_log_size":6217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25441222,"num_objects":6213,"num_object_clones":0,"num_object_copies":12426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4315,"num_write_kb":17260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6158","reported_seq":6231,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.822451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:19.822451+0000","last_peered":"2024-09-11T01:16:19.822451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:19.822451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6158,"log_dups_size":0,"ondisk_log_size":6158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25206784,"num_objects":6154,"num_object_clones":0,"num_object_copies":12308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2149}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6188","reported_seq":6223,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:19.790088+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:19.790088+0000","last_peered":"2024-09-11T01:16:19.790088+0000","last_clean":"2024-09-11T01:16:19.790088+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:19.790088+0000","last_undegraded":"2024-09-11T01:16:19.790088+0000","last_fullsized":"2024-09-11T01:16:19.790088+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6188,"log_dups_size":0,"ondisk_log_size":6188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25346048,"num_objects":6188,"num_object_clones":0,"num_object_copies":12376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6188,"num_write_kb":24752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6205","reported_seq":6316,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.142560+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:20.142560+0000","last_peered":"2024-09-11T01:16:20.142560+0000","last_clean":"2024-09-11T01:16:20.142560+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:20.142560+0000","last_undegraded":"2024-09-11T01:16:20.142560+0000","last_fullsized":"2024-09-11T01:16:20.142560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25387491,"num_objects":6199,"num_object_clones":0,"num_object_copies":12398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6199,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3003,"num_write_kb":12012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6243","reported_seq":6367,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.134618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:20.134618+0000","last_peered":"2024-09-11T01:16:20.134618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:20.134618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6243,"log_dups_size":0,"ondisk_log_size":6243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25551331,"num_objects":6239,"num_object_clones":0,"num_object_copies":12478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2731,"num_write_kb":10924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6377","reported_seq":6500,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.081018+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:20.081018+0000","last_peered":"2024-09-11T01:16:20.081018+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:20.081018+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6377,"log_dups_size":0,"ondisk_log_size":6377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26096582,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2090},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6225","reported_seq":6332,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.125171+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:20.125171+0000","last_peered":"2024-09-11T01:16:20.125171+0000","last_clean":"2024-09-11T01:16:20.125171+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:20.125171+0000","last_undegraded":"2024-09-11T01:16:20.125171+0000","last_fullsized":"2024-09-11T01:16:20.125171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6225,"log_dups_size":0,"ondisk_log_size":6225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3178,"num_write_kb":12712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6218","reported_seq":9271,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:19.872584+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:19.872584+0000","last_peered":"2024-09-11T01:16:19.872584+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:19.872584+0000","last_undegraded":"2024-09-11T01:16:19.872584+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6218,"log_dups_size":0,"ondisk_log_size":6218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25440739,"num_objects":6212,"num_object_clones":0,"num_object_copies":12424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":6212,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4085,"num_write_kb":16340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1490,"num_bytes_recovered":6103040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2634},{"shards":"2","objects":3578}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6281","reported_seq":6400,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.065661+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:20.065661+0000","last_peered":"2024-09-11T01:16:20.065661+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:20.065661+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6281,"log_dups_size":0,"ondisk_log_size":6281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2168},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6131","reported_seq":6213,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.800258+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:19.800258+0000","last_peered":"2024-09-11T01:16:19.800258+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:19.800258+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6131,"log_dups_size":0,"ondisk_log_size":6131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25092579,"num_objects":6127,"num_object_clones":0,"num_object_copies":12254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3900,"num_write_kb":15600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6283","reported_seq":6364,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:19.840342+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:19.840342+0000","last_peered":"2024-09-11T01:16:19.840342+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:19.840342+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710592,"num_objects":6277,"num_object_clones":0,"num_object_copies":12554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6265","reported_seq":6385,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.133670+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:20.133670+0000","last_peered":"2024-09-11T01:16:20.133670+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:20.133670+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6265,"log_dups_size":0,"ondisk_log_size":6265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25653248,"num_objects":6263,"num_object_clones":0,"num_object_copies":12526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2099},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6316","reported_seq":6440,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.090714+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:20.090714+0000","last_peered":"2024-09-11T01:16:20.090714+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:20.090714+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2709,"num_write_kb":10836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2116},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6289","reported_seq":6408,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.098560+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:20.098560+0000","last_peered":"2024-09-11T01:16:20.098560+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:20.098560+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6289,"log_dups_size":0,"ondisk_log_size":6289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25743360,"num_objects":6285,"num_object_clones":0,"num_object_copies":12570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2137},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6191","reported_seq":6316,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.038439+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:20.038439+0000","last_peered":"2024-09-11T01:16:20.038439+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:20.038439+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6191,"log_dups_size":0,"ondisk_log_size":6191,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25310150,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2848,"num_write_kb":11392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2027},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408216286,"num_objects":99671,"num_object_clones":0,"num_object_copies":199342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2634,"num_objects_unfound":0,"num_objects_dirty":99671,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":58806,"num_write_kb":235224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1491,"num_bytes_recovered":6107136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":826753024,"data_stored":826597665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99743,"ondisk_log_size":99743,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503400,"kb_used_data":331480,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93868440,"statfs":{"total":96636764160,"available":96121282560,"internally_reserved":0,"allocated":339435520,"data_stored":339178616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789966,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253996,"kb_used_data":109852,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94117844,"statfs":{"total":96636764160,"available":96376672256,"internally_reserved":0,"allocated":112488448,"data_stored":112180498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549200,"kb_used_data":364864,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93822640,"statfs":{"total":96636764160,"available":96074383360,"internally_reserved":0,"allocated":373620736,"data_stored":373359755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":373776384,"data_stored":373740254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113577984,"data_stored":113494885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339398656,"data_stored":339362526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:26.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:26.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'6156", 'reported_seq': 6219, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.058912+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:20.058912+0000', 'last_peered': '2024-09-11T01:16:20.058912+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:20.058912+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 6156, 'log_dups_size': 0, 'ondisk_log_size': 6156, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25214976, 'num_objects': 6156, 'num_object_clones': 0, 'num_object_copies': 12312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6156, 'num_write_kb': 24624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2033}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:26.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'6158", 'reported_seq': 6231, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:19.822451+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:19.822451+0000', 'last_peered': '2024-09-11T01:16:19.822451+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:19.822451+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 6158, 'log_dups_size': 0, 'ondisk_log_size': 6158, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25206784, 'num_objects': 6154, 'num_object_clones': 0, 'num_object_copies': 12308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4219, 'num_write_kb': 16876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 2149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:26.462 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:26.462 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'6243", 'reported_seq': 6367, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.134618+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:20.134618+0000', 'last_peered': '2024-09-11T01:16:20.134618+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:20.134618+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 6243, 'log_dups_size': 0, 'ondisk_log_size': 6243, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25551331, 'num_objects': 6239, 'num_object_clones': 0, 'num_object_copies': 12478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6239, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2731, 'num_write_kb': 10924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2124}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'6377", 'reported_seq': 6500, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.081018+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:20.081018+0000', 'last_peered': '2024-09-11T01:16:20.081018+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:20.081018+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 6377, 'log_dups_size': 0, 'ondisk_log_size': 6377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26096582, 'num_objects': 6373, 'num_object_clones': 0, 'num_object_copies': 12746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6373, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3130, 'num_write_kb': 12520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2090}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'6218", 'reported_seq': 9271, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:19.872584+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:19.872584+0000', 'last_peered': '2024-09-11T01:16:19.872584+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:19.872584+0000', 'last_undegraded': '2024-09-11T01:16:19.872584+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 6218, 'log_dups_size': 0, 'ondisk_log_size': 6218, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25440739, 'num_objects': 6212, 'num_object_clones': 0, 'num_object_copies': 12424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2634, 'num_objects_unfound': 0, 'num_objects_dirty': 6212, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4085, 'num_write_kb': 16340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1490, 'num_bytes_recovered': 6103040, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2634}, {'shards': '2', 'objects': 3578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'6281", 'reported_seq': 6400, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.065661+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:20.065661+0000', 'last_peered': '2024-09-11T01:16:20.065661+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:20.065661+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 6281, 'log_dups_size': 0, 'ondisk_log_size': 6281, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25694208, 'num_objects': 6273, 'num_object_clones': 0, 'num_object_copies': 12546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2645, 'num_write_kb': 10580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2168}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'6131", 'reported_seq': 6213, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:19.800258+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:19.800258+0000', 'last_peered': '2024-09-11T01:16:19.800258+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:19.800258+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 6131, 'log_dups_size': 0, 'ondisk_log_size': 6131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25092579, 'num_objects': 6127, 'num_object_clones': 0, 'num_object_copies': 12254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6127, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3900, 'num_write_kb': 15600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 2100}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'6283", 'reported_seq': 6364, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:19.840342+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:19.840342+0000', 'last_peered': '2024-09-11T01:16:19.840342+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:19.840342+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 6283, 'log_dups_size': 0, 'ondisk_log_size': 6283, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25710592, 'num_objects': 6277, 'num_object_clones': 0, 'num_object_copies': 12554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3920, 'num_write_kb': 15680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 2100}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:26.463 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'6265", 'reported_seq': 6385, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.133670+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:20.133670+0000', 'last_peered': '2024-09-11T01:16:20.133670+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:20.133670+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 6265, 'log_dups_size': 0, 'ondisk_log_size': 6265, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25653248, 'num_objects': 6263, 'num_object_clones': 0, 'num_object_copies': 12526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6263, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2955, 'num_write_kb': 11820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2099}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'6316", 'reported_seq': 6440, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.090714+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:20.090714+0000', 'last_peered': '2024-09-11T01:16:20.090714+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:20.090714+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 6316, 'log_dups_size': 0, 'ondisk_log_size': 6316, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25837568, 'num_objects': 6308, 'num_object_clones': 0, 'num_object_copies': 12616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2709, 'num_write_kb': 10836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2116}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'6289", 'reported_seq': 6408, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.098560+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:20.098560+0000', 'last_peered': '2024-09-11T01:16:20.098560+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:20.098560+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 6289, 'log_dups_size': 0, 'ondisk_log_size': 6289, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25743360, 'num_objects': 6285, 'num_object_clones': 0, 'num_object_copies': 12570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2824, 'num_write_kb': 11296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2137}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:26.464 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'6191", 'reported_seq': 6316, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.038439+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:20.038439+0000', 'last_peered': '2024-09-11T01:16:20.038439+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:20.038439+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 6191, 'log_dups_size': 0, 'ondisk_log_size': 6191, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25310150, 'num_objects': 6181, 'num_object_clones': 0, 'num_object_copies': 12362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6181, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2848, 'num_write_kb': 11392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2027}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:26.464 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:27.169 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:27.170 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/204644 objects degraded (22.238%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":187,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102322,"data_bytes":419066609,"bytes_used":1343000576,"bytes_avail":288567291904,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":204644,"degraded_ratio":0.22237641953832021,"misplaced_objects":2496,"misplaced_total":204644,"misplaced_ratio":0.012196790524031977,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":137866,"recovering_keys_per_sec":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"write_bytes_sec":2747778,"read_op_per_sec":0,"write_op_per_sec":670},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:27.194 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:27.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:27.337 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 111 20 103400 103380 3.63724 1.84375 0.0173132 0.021469 2024-09-11T01:16:27.764 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:27.764 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:27.792 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-11T01:16:26.437593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419066609,"num_objects":102322,"num_object_clones":0,"num_object_copies":204644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2496,"num_objects_unfound":0,"num_objects_dirty":102322,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":61457,"num_write_kb":245822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1630,"num_bytes_recovered":6672403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102395,"ondisk_log_size":102395,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1311524,"kb_used_data":819444,"kb_used_omap":12,"kb_used_meta":492019,"kb_avail":281803996,"statfs":{"total":289910292480,"available":288567291904,"internally_reserved":0,"allocated":839110656,"data_stored":838284821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":503827463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,8,17,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32980992,"num_objects":8052,"num_object_clones":0,"num_object_copies":16104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-404,"num_objects_unfound":0,"num_objects_dirty":8052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8052,"num_write_kb":32208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1654784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002786"},"pg_stats":[{"pgid":"2.f","version":"39'6306","reported_seq":6368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.170804+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:25.170804+0000","last_peered":"2024-09-11T01:16:25.170804+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:25.170804+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6306,"log_dups_size":0,"ondisk_log_size":6306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25829376,"num_objects":6306,"num_object_clones":0,"num_object_copies":12612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6306,"num_write_kb":25224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2183},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6398","reported_seq":6468,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:24.834589+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:24.834589+0000","last_peered":"2024-09-11T01:16:24.834589+0000","last_clean":"2024-09-11T01:16:24.834589+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:24.834589+0000","last_undegraded":"2024-09-11T01:16:24.834589+0000","last_fullsized":"2024-09-11T01:16:24.834589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6398,"log_dups_size":0,"ondisk_log_size":6398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26182598,"num_objects":6394,"num_object_clones":0,"num_object_copies":12788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6394,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4496,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6331","reported_seq":6404,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:24.818900+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:24.818900+0000","last_peered":"2024-09-11T01:16:24.818900+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:24.818900+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6331,"log_dups_size":0,"ondisk_log_size":6331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25915392,"num_objects":6327,"num_object_clones":0,"num_object_copies":12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4392,"num_write_kb":17568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6342","reported_seq":6377,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:24.878125+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:24.878125+0000","last_peered":"2024-09-11T01:16:24.878125+0000","last_clean":"2024-09-11T01:16:24.878125+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:24.878125+0000","last_undegraded":"2024-09-11T01:16:24.878125+0000","last_fullsized":"2024-09-11T01:16:24.878125+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6342,"log_dups_size":0,"ondisk_log_size":6342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6371","reported_seq":6483,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:25.138959+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:25.138959+0000","last_peered":"2024-09-11T01:16:25.138959+0000","last_clean":"2024-09-11T01:16:25.138959+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:25.138959+0000","last_undegraded":"2024-09-11T01:16:25.138959+0000","last_fullsized":"2024-09-11T01:16:25.138959+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6371,"log_dups_size":0,"ondisk_log_size":6371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26067427,"num_objects":6365,"num_object_clones":0,"num_object_copies":12730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6407","reported_seq":6531,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.159372+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:25.159372+0000","last_peered":"2024-09-11T01:16:25.159372+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:25.159372+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6407,"log_dups_size":0,"ondisk_log_size":6407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26223075,"num_objects":6403,"num_object_clones":0,"num_object_copies":12806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2895,"num_write_kb":11580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2288},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6558","reported_seq":6681,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.143823+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:25.143823+0000","last_peered":"2024-09-11T01:16:25.143823+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:25.143823+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6558,"log_dups_size":0,"ondisk_log_size":6558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26837958,"num_objects":6554,"num_object_clones":0,"num_object_copies":13108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6554,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3311,"num_write_kb":13244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2271},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6384","reported_seq":6494,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:25.125900+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:25.125900+0000","last_peered":"2024-09-11T01:16:25.125900+0000","last_clean":"2024-09-11T01:16:25.125900+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:25.125900+0000","last_undegraded":"2024-09-11T01:16:25.125900+0000","last_fullsized":"2024-09-11T01:16:25.125900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6384,"log_dups_size":0,"ondisk_log_size":6384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26140672,"num_objects":6382,"num_object_clones":0,"num_object_copies":12764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3337,"num_write_kb":13348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6380","reported_seq":9707,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:24.887086+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:24.887086+0000","last_peered":"2024-09-11T01:16:24.887086+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:24.887086+0000","last_undegraded":"2024-09-11T01:16:24.887086+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6380,"log_dups_size":0,"ondisk_log_size":6380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26104291,"num_objects":6374,"num_object_clones":0,"num_object_copies":12748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2496,"num_objects_unfound":0,"num_objects_dirty":6374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4247,"num_write_kb":16988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1628,"num_bytes_recovered":6668288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2496},{"shards":"2","objects":3878}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990365+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990365+0000","last_peered":"2024-09-11T01:15:19.990365+0000","last_clean":"2024-09-11T01:15:19.990365+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990365+0000","last_undegraded":"2024-09-11T01:15:19.990365+0000","last_fullsized":"2024-09-11T01:15:19.990365+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6446","reported_seq":6565,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.109352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:25.109352+0000","last_peered":"2024-09-11T01:16:25.109352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:25.109352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6446,"log_dups_size":0,"ondisk_log_size":6446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26370048,"num_objects":6438,"num_object_clones":0,"num_object_copies":12876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2810,"num_write_kb":11240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2333},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6283","reported_seq":6367,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:24.863014+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:24.863014+0000","last_peered":"2024-09-11T01:16:24.863014+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:24.863014+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6283,"log_dups_size":0,"ondisk_log_size":6283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25715171,"num_objects":6279,"num_object_clones":0,"num_object_copies":12558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4052,"num_write_kb":16208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6453","reported_seq":6536,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:24.912898+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:24.912898+0000","last_peered":"2024-09-11T01:16:24.912898+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:24.912898+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6453,"log_dups_size":0,"ondisk_log_size":6453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26406912,"num_objects":6447,"num_object_clones":0,"num_object_copies":12894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4090,"num_write_kb":16360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2270}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6440","reported_seq":6560,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.146329+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:25.146329+0000","last_peered":"2024-09-11T01:16:25.146329+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:25.146329+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6440,"log_dups_size":0,"ondisk_log_size":6440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26370048,"num_objects":6438,"num_object_clones":0,"num_object_copies":12876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3130,"num_write_kb":12520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2274},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6489","reported_seq":6613,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.168115+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:25.168115+0000","last_peered":"2024-09-11T01:16:25.168115+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:25.168115+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6489,"log_dups_size":0,"ondisk_log_size":6489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26546176,"num_objects":6481,"num_object_clones":0,"num_object_copies":12962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2882,"num_write_kb":11528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2289},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031437+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031437+0000","last_peered":"2024-09-11T01:15:20.031437+0000","last_clean":"2024-09-11T01:15:20.031437+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031437+0000","last_undegraded":"2024-09-11T01:15:20.031437+0000","last_fullsized":"2024-09-11T01:15:20.031437+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6463","reported_seq":6585,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.094034+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:25.094034+0000","last_peered":"2024-09-11T01:16:25.094034+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:25.094034+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6463,"log_dups_size":0,"ondisk_log_size":6463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26456064,"num_objects":6459,"num_object_clones":0,"num_object_copies":12918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2998,"num_write_kb":11992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2311},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6341","reported_seq":6465,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.118868+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:25.118868+0000","last_peered":"2024-09-11T01:16:25.118868+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:25.118868+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6341,"log_dups_size":0,"ondisk_log_size":6341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25924550,"num_objects":6331,"num_object_clones":0,"num_object_copies":12662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6331,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2998,"num_write_kb":11992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2177},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419066590,"num_objects":102320,"num_object_clones":0,"num_object_copies":204640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2496,"num_objects_unfound":0,"num_objects_dirty":102320,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":61455,"num_write_kb":245820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1629,"num_bytes_recovered":6672384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":840368128,"data_stored":840212769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102392,"ondisk_log_size":102392,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512852,"kb_used_data":336772,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93858988,"statfs":{"total":96636764160,"available":96111603712,"internally_reserved":0,"allocated":344854528,"data_stored":344597624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789966,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":253996,"kb_used_data":109852,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94117844,"statfs":{"total":96636764160,"available":96376672256,"internally_reserved":0,"allocated":112488448,"data_stored":112180498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544676,"kb_used_data":372820,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93827164,"statfs":{"total":96636764160,"available":96079015936,"internally_reserved":0,"allocated":381767680,"data_stored":381506699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,10,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":381968384,"data_stored":381932254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113577984,"data_stored":113494885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344821760,"data_stored":344785630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:28.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 112 20 103930 103910 3.62325 2.07031 0.0640268 0.0215549 2024-09-11T01:16:29.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 113 20 104471 104451 3.60989 2.11328 0.0653102 0.0216346 2024-09-11T01:16:30.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 114 20 105011 104991 3.59672 2.10938 0.0134206 0.0217144 2024-09-11T01:16:30.795 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:31.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 115 20 105519 105499 3.5827 1.98438 0.0266816 0.0217981 2024-09-11T01:16:31.374 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:31.374 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:31.398 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-11T01:16:30.438692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":429601521,"num_objects":104894,"num_object_clones":0,"num_object_copies":209788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":104894,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":64029,"num_write_kb":256110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":7258131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104967,"ondisk_log_size":104967,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1363484,"kb_used_data":840876,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281752036,"statfs":{"total":289910292480,"available":288514084864,"internally_reserved":0,"allocated":861057024,"data_stored":860231189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":535088135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,5,2,8,16,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32387072,"num_objects":7907,"num_object_clones":0,"num_object_copies":15814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-420,"num_objects_unfound":0,"num_objects_dirty":7907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7907,"num_write_kb":31628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002965"},"pg_stats":[{"pgid":"2.f","version":"39'6467","reported_seq":6531,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.210397+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:30.210397+0000","last_peered":"2024-09-11T01:16:30.210397+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:30.210397+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6568","reported_seq":6638,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:29.902702+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:29.902702+0000","last_peered":"2024-09-11T01:16:29.902702+0000","last_clean":"2024-09-11T01:16:29.902702+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:29.902702+0000","last_undegraded":"2024-09-11T01:16:29.902702+0000","last_fullsized":"2024-09-11T01:16:29.902702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26878918,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6482","reported_seq":6554,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.944991+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:29.944991+0000","last_peered":"2024-09-11T01:16:29.944991+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:29.944991+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6482,"log_dups_size":0,"ondisk_log_size":6482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533888,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6508","reported_seq":6543,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:29.936397+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:29.936397+0000","last_peered":"2024-09-11T01:16:29.936397+0000","last_clean":"2024-09-11T01:16:29.936397+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:29.936397+0000","last_undegraded":"2024-09-11T01:16:29.936397+0000","last_fullsized":"2024-09-11T01:16:29.936397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6508,"log_dups_size":0,"ondisk_log_size":6508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6534","reported_seq":6646,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.146604+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:30.146604+0000","last_peered":"2024-09-11T01:16:30.146604+0000","last_clean":"2024-09-11T01:16:30.146604+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:30.146604+0000","last_undegraded":"2024-09-11T01:16:30.146604+0000","last_fullsized":"2024-09-11T01:16:30.146604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6534,"log_dups_size":0,"ondisk_log_size":6534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735075,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3332,"num_write_kb":13328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6580","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.196170+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:30.196170+0000","last_peered":"2024-09-11T01:16:30.196170+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:30.196170+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931683,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3068,"num_write_kb":12272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2461},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6710","reported_seq":6832,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.150572+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:30.150572+0000","last_peered":"2024-09-11T01:16:30.150572+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:30.150572+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460550,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3463,"num_write_kb":13852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2423},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6551","reported_seq":6661,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.190568+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:30.190568+0000","last_peered":"2024-09-11T01:16:30.190568+0000","last_clean":"2024-09-11T01:16:30.190568+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:30.190568+0000","last_undegraded":"2024-09-11T01:16:30.190568+0000","last_fullsized":"2024-09-11T01:16:30.190568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6551,"log_dups_size":0,"ondisk_log_size":6551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3504,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6543","reported_seq":10158,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:29.938174+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:29.938174+0000","last_peered":"2024-09-11T01:16:29.938174+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:29.938174+0000","last_undegraded":"2024-09-11T01:16:29.938174+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":7254016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2353},{"shards":"2","objects":4184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6604","reported_seq":6723,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.155976+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:30.155976+0000","last_peered":"2024-09-11T01:16:30.155976+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:30.155976+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6604,"log_dups_size":0,"ondisk_log_size":6604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2968,"num_write_kb":11872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2491},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6432","reported_seq":6512,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.899720+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:29.899720+0000","last_peered":"2024-09-11T01:16:29.899720+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:29.899720+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6432,"log_dups_size":0,"ondisk_log_size":6432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26325475,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4201,"num_write_kb":16804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2401}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6613","reported_seq":6697,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.880129+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:29.880129+0000","last_peered":"2024-09-11T01:16:29.880129+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:29.880129+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6613,"log_dups_size":0,"ondisk_log_size":6613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27062272,"num_objects":6607,"num_object_clones":0,"num_object_copies":13214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4250,"num_write_kb":17000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6585","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.102774+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:30.102774+0000","last_peered":"2024-09-11T01:16:30.102774+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:30.102774+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6585,"log_dups_size":0,"ondisk_log_size":6585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26963968,"num_objects":6583,"num_object_clones":0,"num_object_copies":13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3275,"num_write_kb":13100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6663","reported_seq":6786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.183247+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:30.183247+0000","last_peered":"2024-09-11T01:16:30.183247+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:30.183247+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6663,"log_dups_size":0,"ondisk_log_size":6663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3056,"num_write_kb":12224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2463},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6628","reported_seq":6752,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.176595+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:30.176595+0000","last_peered":"2024-09-11T01:16:30.176595+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:30.176595+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2476},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6496","reported_seq":6621,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.108363+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:30.108363+0000","last_peered":"2024-09-11T01:16:30.108363+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:30.108363+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3153,"num_write_kb":12612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":429601502,"num_objects":104892,"num_object_clones":0,"num_object_copies":209784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":104892,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":64027,"num_write_kb":256108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":7258112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":862265344,"data_stored":862109985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104964,"ondisk_log_size":104964,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522184,"kb_used_data":341944,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93849656,"statfs":{"total":96636764160,"available":96102047744,"internally_reserved":0,"allocated":350150656,"data_stored":349893752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789967,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266520,"kb_used_data":118216,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94105320,"statfs":{"total":96636764160,"available":96363847680,"internally_reserved":0,"allocated":121053184,"data_stored":120745234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574780,"kb_used_data":380716,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93797060,"statfs":{"total":96636764160,"available":96048189440,"internally_reserved":0,"allocated":389853184,"data_stored":389592203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,0,0,3,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389951488,"data_stored":389915358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122216448,"data_stored":122133349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350097408,"data_stored":350061278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:31.400 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:31.984 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:31.985 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:32.008 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-11T01:16:30.438692+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":429601521,"num_objects":104894,"num_object_clones":0,"num_object_copies":209788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":104894,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":64029,"num_write_kb":256110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":7258131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104967,"ondisk_log_size":104967,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1363484,"kb_used_data":840876,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281752036,"statfs":{"total":289910292480,"available":288514084864,"internally_reserved":0,"allocated":861057024,"data_stored":860231189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":535088135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,5,2,8,16,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32387072,"num_objects":7907,"num_object_clones":0,"num_object_copies":15814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-420,"num_objects_unfound":0,"num_objects_dirty":7907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7907,"num_write_kb":31628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002965"},"pg_stats":[{"pgid":"2.f","version":"39'6467","reported_seq":6531,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.210397+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:30.210397+0000","last_peered":"2024-09-11T01:16:30.210397+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:30.210397+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6568","reported_seq":6638,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:29.902702+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:29.902702+0000","last_peered":"2024-09-11T01:16:29.902702+0000","last_clean":"2024-09-11T01:16:29.902702+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:29.902702+0000","last_undegraded":"2024-09-11T01:16:29.902702+0000","last_fullsized":"2024-09-11T01:16:29.902702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26878918,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6482","reported_seq":6554,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.944991+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:29.944991+0000","last_peered":"2024-09-11T01:16:29.944991+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:29.944991+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6482,"log_dups_size":0,"ondisk_log_size":6482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533888,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6508","reported_seq":6543,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:29.936397+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:29.936397+0000","last_peered":"2024-09-11T01:16:29.936397+0000","last_clean":"2024-09-11T01:16:29.936397+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:29.936397+0000","last_undegraded":"2024-09-11T01:16:29.936397+0000","last_fullsized":"2024-09-11T01:16:29.936397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6508,"log_dups_size":0,"ondisk_log_size":6508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6534","reported_seq":6646,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.146604+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:30.146604+0000","last_peered":"2024-09-11T01:16:30.146604+0000","last_clean":"2024-09-11T01:16:30.146604+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:30.146604+0000","last_undegraded":"2024-09-11T01:16:30.146604+0000","last_fullsized":"2024-09-11T01:16:30.146604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6534,"log_dups_size":0,"ondisk_log_size":6534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735075,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3332,"num_write_kb":13328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6580","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.196170+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:30.196170+0000","last_peered":"2024-09-11T01:16:30.196170+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:30.196170+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931683,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3068,"num_write_kb":12272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2461},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6710","reported_seq":6832,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.150572+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:30.150572+0000","last_peered":"2024-09-11T01:16:30.150572+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:30.150572+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460550,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3463,"num_write_kb":13852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2423},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6551","reported_seq":6661,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.190568+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:30.190568+0000","last_peered":"2024-09-11T01:16:30.190568+0000","last_clean":"2024-09-11T01:16:30.190568+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:30.190568+0000","last_undegraded":"2024-09-11T01:16:30.190568+0000","last_fullsized":"2024-09-11T01:16:30.190568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6551,"log_dups_size":0,"ondisk_log_size":6551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3504,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6543","reported_seq":10158,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:29.938174+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:29.938174+0000","last_peered":"2024-09-11T01:16:29.938174+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:29.938174+0000","last_undegraded":"2024-09-11T01:16:29.938174+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":7254016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2353},{"shards":"2","objects":4184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6604","reported_seq":6723,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.155976+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:30.155976+0000","last_peered":"2024-09-11T01:16:30.155976+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:30.155976+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6604,"log_dups_size":0,"ondisk_log_size":6604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2968,"num_write_kb":11872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2491},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6432","reported_seq":6512,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.899720+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:29.899720+0000","last_peered":"2024-09-11T01:16:29.899720+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:29.899720+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6432,"log_dups_size":0,"ondisk_log_size":6432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26325475,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4201,"num_write_kb":16804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2401}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6613","reported_seq":6697,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.880129+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:29.880129+0000","last_peered":"2024-09-11T01:16:29.880129+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:29.880129+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6613,"log_dups_size":0,"ondisk_log_size":6613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27062272,"num_objects":6607,"num_object_clones":0,"num_object_copies":13214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4250,"num_write_kb":17000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6585","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.102774+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:30.102774+0000","last_peered":"2024-09-11T01:16:30.102774+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:30.102774+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6585,"log_dups_size":0,"ondisk_log_size":6585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26963968,"num_objects":6583,"num_object_clones":0,"num_object_copies":13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3275,"num_write_kb":13100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6663","reported_seq":6786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.183247+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:30.183247+0000","last_peered":"2024-09-11T01:16:30.183247+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:30.183247+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6663,"log_dups_size":0,"ondisk_log_size":6663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3056,"num_write_kb":12224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2463},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6628","reported_seq":6752,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.176595+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:30.176595+0000","last_peered":"2024-09-11T01:16:30.176595+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:30.176595+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2476},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6496","reported_seq":6621,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.108363+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:30.108363+0000","last_peered":"2024-09-11T01:16:30.108363+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:30.108363+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3153,"num_write_kb":12612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":429601502,"num_objects":104892,"num_object_clones":0,"num_object_copies":209784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":104892,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":64027,"num_write_kb":256108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":7258112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":862265344,"data_stored":862109985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104964,"ondisk_log_size":104964,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522184,"kb_used_data":341944,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93849656,"statfs":{"total":96636764160,"available":96102047744,"internally_reserved":0,"allocated":350150656,"data_stored":349893752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789967,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266520,"kb_used_data":118216,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94105320,"statfs":{"total":96636764160,"available":96363847680,"internally_reserved":0,"allocated":121053184,"data_stored":120745234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574780,"kb_used_data":380716,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93797060,"statfs":{"total":96636764160,"available":96048189440,"internally_reserved":0,"allocated":389853184,"data_stored":389592203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,0,0,3,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389951488,"data_stored":389915358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122216448,"data_stored":122133349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350097408,"data_stored":350061278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:32.010 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:32.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'6467", 'reported_seq': 6531, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.210397+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:30.210397+0000', 'last_peered': '2024-09-11T01:16:30.210397+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:30.210397+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 6467, 'log_dups_size': 0, 'ondisk_log_size': 6467, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26488832, 'num_objects': 6467, 'num_object_clones': 0, 'num_object_copies': 12934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6467, 'num_write_kb': 25868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2344}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:32.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'6482", 'reported_seq': 6554, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:29.944991+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:29.944991+0000', 'last_peered': '2024-09-11T01:16:29.944991+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:29.944991+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 6482, 'log_dups_size': 0, 'ondisk_log_size': 6482, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26533888, 'num_objects': 6478, 'num_object_clones': 0, 'num_object_copies': 12956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4543, 'num_write_kb': 18172, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 2473}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:32.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'6580", 'reported_seq': 6705, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.196170+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:30.196170+0000', 'last_peered': '2024-09-11T01:16:30.196170+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:30.196170+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 6580, 'log_dups_size': 0, 'ondisk_log_size': 6580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26931683, 'num_objects': 6576, 'num_object_clones': 0, 'num_object_copies': 13152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6576, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3068, 'num_write_kb': 12272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2461}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:32.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'6710", 'reported_seq': 6832, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.150572+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:30.150572+0000', 'last_peered': '2024-09-11T01:16:30.150572+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:30.150572+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 6710, 'log_dups_size': 0, 'ondisk_log_size': 6710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27460550, 'num_objects': 6706, 'num_object_clones': 0, 'num_object_copies': 13412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6706, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3463, 'num_write_kb': 13852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2423}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:32.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'6543", 'reported_seq': 10158, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:29.938174+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:29.938174+0000', 'last_peered': '2024-09-11T01:16:29.938174+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:29.938174+0000', 'last_undegraded': '2024-09-11T01:16:29.938174+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 6543, 'log_dups_size': 0, 'ondisk_log_size': 6543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26771939, 'num_objects': 6537, 'num_object_clones': 0, 'num_object_copies': 13074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2353, 'num_objects_unfound': 0, 'num_objects_dirty': 6537, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4410, 'num_write_kb': 17640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1771, 'num_bytes_recovered': 7254016, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2353}, {'shards': '2', 'objects': 4184}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:32.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'6604", 'reported_seq': 6723, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.155976+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:30.155976+0000', 'last_peered': '2024-09-11T01:16:30.155976+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:30.155976+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 6604, 'log_dups_size': 0, 'ondisk_log_size': 6604, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27017216, 'num_objects': 6596, 'num_object_clones': 0, 'num_object_copies': 13192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2968, 'num_write_kb': 11872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2491}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:32.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'6432", 'reported_seq': 6512, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:29.899720+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:29.899720+0000', 'last_peered': '2024-09-11T01:16:29.899720+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:29.899720+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 6432, 'log_dups_size': 0, 'ondisk_log_size': 6432, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26325475, 'num_objects': 6428, 'num_object_clones': 0, 'num_object_copies': 12856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6428, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4201, 'num_write_kb': 16804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 2401}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:32.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'6613", 'reported_seq': 6697, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:29.880129+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:29.880129+0000', 'last_peered': '2024-09-11T01:16:29.880129+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:29.880129+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 6613, 'log_dups_size': 0, 'ondisk_log_size': 6613, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27062272, 'num_objects': 6607, 'num_object_clones': 0, 'num_object_copies': 13214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6607, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4250, 'num_write_kb': 17000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 2430}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:32.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'6585", 'reported_seq': 6705, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.102774+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:30.102774+0000', 'last_peered': '2024-09-11T01:16:30.102774+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:30.102774+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 6585, 'log_dups_size': 0, 'ondisk_log_size': 6585, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26963968, 'num_objects': 6583, 'num_object_clones': 0, 'num_object_copies': 13166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6583, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3275, 'num_write_kb': 13100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2419}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:32.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'6663", 'reported_seq': 6786, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.183247+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:30.183247+0000', 'last_peered': '2024-09-11T01:16:30.183247+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:30.183247+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 6663, 'log_dups_size': 0, 'ondisk_log_size': 6663, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27258880, 'num_objects': 6655, 'num_object_clones': 0, 'num_object_copies': 13310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3056, 'num_write_kb': 12224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2463}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:32.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'6628", 'reported_seq': 6752, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.176595+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:30.176595+0000', 'last_peered': '2024-09-11T01:16:30.176595+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:30.176595+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 6628, 'log_dups_size': 0, 'ondisk_log_size': 6628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27131904, 'num_objects': 6624, 'num_object_clones': 0, 'num_object_copies': 13248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3163, 'num_write_kb': 12652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2476}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:32.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'6496", 'reported_seq': 6621, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.108363+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:30.108363+0000', 'last_peered': '2024-09-11T01:16:30.108363+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:30.108363+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 6496, 'log_dups_size': 0, 'ondisk_log_size': 6496, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26559430, 'num_objects': 6486, 'num_object_clones': 0, 'num_object_copies': 12972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6486, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3153, 'num_write_kb': 12612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2332}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:32.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 116 20 105991 105971 3.56771 1.84375 0.0317864 0.0218901 2024-09-11T01:16:32.738 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:32.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/209788 objects degraded (21.692%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":193,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":104894,"data_bytes":429601521,"bytes_used":1396207616,"bytes_avail":288514084864,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":209788,"degraded_ratio":0.21692375159685015,"misplaced_objects":2353,"misplaced_total":209788,"misplaced_ratio":0.011216084809426659,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":143324,"recovering_keys_per_sec":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"write_bytes_sec":2698255,"read_op_per_sec":0,"write_op_per_sec":658},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:32.768 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:32.769 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:33.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 117 20 106479 106459 3.55351 1.90625 0.04001 0.0219781 2024-09-11T01:16:33.360 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:33.360 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:33.393 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-11T01:16:32.439237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":429601521,"num_objects":104894,"num_object_clones":0,"num_object_copies":209788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":104894,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":64029,"num_write_kb":256110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":7258131,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104967,"ondisk_log_size":104967,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1363484,"kb_used_data":840876,"kb_used_omap":12,"kb_used_meta":522547,"kb_avail":281752036,"statfs":{"total":289910292480,"available":288514084864,"internally_reserved":0,"allocated":861057024,"data_stored":860231189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":535088135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,5,2,8,16,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21385216,"num_objects":5221,"num_object_clones":0,"num_object_copies":10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5221,"num_write_kb":20884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":281,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003066"},"pg_stats":[{"pgid":"2.f","version":"39'6467","reported_seq":6531,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.210397+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:30.210397+0000","last_peered":"2024-09-11T01:16:30.210397+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:30.210397+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6467,"log_dups_size":0,"ondisk_log_size":6467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6568","reported_seq":6638,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:29.902702+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:29.902702+0000","last_peered":"2024-09-11T01:16:29.902702+0000","last_clean":"2024-09-11T01:16:29.902702+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:29.902702+0000","last_undegraded":"2024-09-11T01:16:29.902702+0000","last_fullsized":"2024-09-11T01:16:29.902702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26878918,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4666,"num_write_kb":18664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6482","reported_seq":6554,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.944991+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:29.944991+0000","last_peered":"2024-09-11T01:16:29.944991+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:29.944991+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6482,"log_dups_size":0,"ondisk_log_size":6482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26533888,"num_objects":6478,"num_object_clones":0,"num_object_copies":12956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4543,"num_write_kb":18172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6508","reported_seq":6543,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:29.936397+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:29.936397+0000","last_peered":"2024-09-11T01:16:29.936397+0000","last_clean":"2024-09-11T01:16:29.936397+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:29.936397+0000","last_undegraded":"2024-09-11T01:16:29.936397+0000","last_fullsized":"2024-09-11T01:16:29.936397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6508,"log_dups_size":0,"ondisk_log_size":6508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26656768,"num_objects":6508,"num_object_clones":0,"num_object_copies":13016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6534","reported_seq":6646,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.146604+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:30.146604+0000","last_peered":"2024-09-11T01:16:30.146604+0000","last_clean":"2024-09-11T01:16:30.146604+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:30.146604+0000","last_undegraded":"2024-09-11T01:16:30.146604+0000","last_fullsized":"2024-09-11T01:16:30.146604+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6534,"log_dups_size":0,"ondisk_log_size":6534,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735075,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3332,"num_write_kb":13328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6580","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.196170+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:30.196170+0000","last_peered":"2024-09-11T01:16:30.196170+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:30.196170+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6580,"log_dups_size":0,"ondisk_log_size":6580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26931683,"num_objects":6576,"num_object_clones":0,"num_object_copies":13152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3068,"num_write_kb":12272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2461},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6710","reported_seq":6832,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.150572+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:30.150572+0000","last_peered":"2024-09-11T01:16:30.150572+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:30.150572+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6710,"log_dups_size":0,"ondisk_log_size":6710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27460550,"num_objects":6706,"num_object_clones":0,"num_object_copies":13412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3463,"num_write_kb":13852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2423},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6551","reported_seq":6661,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.190568+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:30.190568+0000","last_peered":"2024-09-11T01:16:30.190568+0000","last_clean":"2024-09-11T01:16:30.190568+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:30.190568+0000","last_undegraded":"2024-09-11T01:16:30.190568+0000","last_fullsized":"2024-09-11T01:16:30.190568+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6551,"log_dups_size":0,"ondisk_log_size":6551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26824704,"num_objects":6549,"num_object_clones":0,"num_object_copies":13098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3504,"num_write_kb":14016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6543","reported_seq":10158,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:29.938174+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:29.938174+0000","last_peered":"2024-09-11T01:16:29.938174+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:29.938174+0000","last_undegraded":"2024-09-11T01:16:29.938174+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6543,"log_dups_size":0,"ondisk_log_size":6543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26771939,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":7254016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2353},{"shards":"2","objects":4184}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6604","reported_seq":6723,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.155976+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:30.155976+0000","last_peered":"2024-09-11T01:16:30.155976+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:30.155976+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6604,"log_dups_size":0,"ondisk_log_size":6604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2968,"num_write_kb":11872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2491},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6432","reported_seq":6512,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.899720+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:29.899720+0000","last_peered":"2024-09-11T01:16:29.899720+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:29.899720+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6432,"log_dups_size":0,"ondisk_log_size":6432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26325475,"num_objects":6428,"num_object_clones":0,"num_object_copies":12856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4201,"num_write_kb":16804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2401}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6613","reported_seq":6697,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.880129+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:29.880129+0000","last_peered":"2024-09-11T01:16:29.880129+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:29.880129+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6613,"log_dups_size":0,"ondisk_log_size":6613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27062272,"num_objects":6607,"num_object_clones":0,"num_object_copies":13214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4250,"num_write_kb":17000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6585","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.102774+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:30.102774+0000","last_peered":"2024-09-11T01:16:30.102774+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:30.102774+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6585,"log_dups_size":0,"ondisk_log_size":6585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26963968,"num_objects":6583,"num_object_clones":0,"num_object_copies":13166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3275,"num_write_kb":13100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2419},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6663","reported_seq":6786,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.183247+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:30.183247+0000","last_peered":"2024-09-11T01:16:30.183247+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:30.183247+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6663,"log_dups_size":0,"ondisk_log_size":6663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3056,"num_write_kb":12224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2463},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6628","reported_seq":6752,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.176595+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:30.176595+0000","last_peered":"2024-09-11T01:16:30.176595+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:30.176595+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2476},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6496","reported_seq":6621,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.108363+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:30.108363+0000","last_peered":"2024-09-11T01:16:30.108363+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:30.108363+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26559430,"num_objects":6486,"num_object_clones":0,"num_object_copies":12972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3153,"num_write_kb":12612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":429601502,"num_objects":104892,"num_object_clones":0,"num_object_copies":209784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2353,"num_objects_unfound":0,"num_objects_dirty":104892,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":64027,"num_write_kb":256108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1772,"num_bytes_recovered":7258112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":862265344,"data_stored":862109985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104964,"ondisk_log_size":104964,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522184,"kb_used_data":341944,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93849656,"statfs":{"total":96636764160,"available":96102047744,"internally_reserved":0,"allocated":350150656,"data_stored":349893752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":184544942},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789967,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266520,"kb_used_data":118216,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94105320,"statfs":{"total":96636764160,"available":96363847680,"internally_reserved":0,"allocated":121053184,"data_stored":120745234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574780,"kb_used_data":380716,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93797060,"statfs":{"total":96636764160,"available":96048189440,"internally_reserved":0,"allocated":389853184,"data_stored":389592203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,0,0,3,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":389951488,"data_stored":389915358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122216448,"data_stored":122133349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350097408,"data_stored":350061278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:34.338 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 118 20 107026 107006 3.54151 2.13672 0.0730397 0.0220521 2024-09-11T01:16:35.339 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 119 20 107560 107540 3.52924 2.08594 0.0267588 0.0221287 2024-09-11T01:16:36.339 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:16:36.339005+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0222032 2024-09-11T01:16:36.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:16:36.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 120 20 108102 108082 3.51748 2.11719 0.0570366 0.0222032 2024-09-11T01:16:36.395 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:36.968 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:36.970 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:36.991 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-11T01:16:36.440204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440103665,"num_objects":107458,"num_object_clones":0,"num_object_copies":214916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":107458,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":66593,"num_write_kb":266366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1919,"num_bytes_recovered":7856147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107531,"ondisk_log_size":107531,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443068,"kb_used_data":862156,"kb_used_omap":12,"kb_used_meta":580851,"kb_avail":281672452,"statfs":{"total":289910292480,"available":288432590848,"internally_reserved":0,"allocated":882847744,"data_stored":882021909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":594791431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,8,14,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7785,"num_write_kb":31140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003056"},"pg_stats":[{"pgid":"2.f","version":"39'6619","reported_seq":6681,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.262042+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:35.262042+0000","last_peered":"2024-09-11T01:16:35.262042+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:35.262042+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6619,"log_dups_size":0,"ondisk_log_size":6619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":26476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2496},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6735","reported_seq":6805,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.954146+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:34.954146+0000","last_peered":"2024-09-11T01:16:34.954146+0000","last_clean":"2024-09-11T01:16:34.954146+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:34.954146+0000","last_undegraded":"2024-09-11T01:16:34.954146+0000","last_fullsized":"2024-09-11T01:16:34.954146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6735,"log_dups_size":0,"ondisk_log_size":6735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27562950,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6632","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.937059+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:34.937059+0000","last_peered":"2024-09-11T01:16:34.937059+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:34.937059+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4693,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2623}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6670","reported_seq":6705,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.923692+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:34.923692+0000","last_peered":"2024-09-11T01:16:34.923692+0000","last_clean":"2024-09-11T01:16:34.923692+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:34.923692+0000","last_undegraded":"2024-09-11T01:16:34.923692+0000","last_fullsized":"2024-09-11T01:16:34.923692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":26680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6708","reported_seq":6820,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.163390+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:35.163390+0000","last_peered":"2024-09-11T01:16:35.163390+0000","last_clean":"2024-09-11T01:16:35.163390+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:35.163390+0000","last_undegraded":"2024-09-11T01:16:35.163390+0000","last_fullsized":"2024-09-11T01:16:35.163390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6708,"log_dups_size":0,"ondisk_log_size":6708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27447779,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3506,"num_write_kb":14024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6733","reported_seq":6858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.166772+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:35.166772+0000","last_peered":"2024-09-11T01:16:35.166772+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:35.166772+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27558371,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2614},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6869","reported_seq":6993,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.145632+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:35.145632+0000","last_peered":"2024-09-11T01:16:35.145632+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:35.145632+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28111814,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2582},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6711","reported_seq":6821,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.166870+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:35.166870+0000","last_peered":"2024-09-11T01:16:35.166870+0000","last_clean":"2024-09-11T01:16:35.166870+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:35.166870+0000","last_undegraded":"2024-09-11T01:16:35.166870+0000","last_fullsized":"2024-09-11T01:16:35.166870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6711,"log_dups_size":0,"ondisk_log_size":6711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27480064,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6704","reported_seq":10608,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:34.952722+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:34.952722+0000","last_peered":"2024-09-11T01:16:34.952722+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:34.952722+0000","last_undegraded":"2024-09-11T01:16:34.952722+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6704,"log_dups_size":0,"ondisk_log_size":6704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27431395,"num_objects":6698,"num_object_clones":0,"num_object_copies":13396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":6698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4571,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1917,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2207},{"shards":"2","objects":4491}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6747","reported_seq":6865,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.218868+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:35.218868+0000","last_peered":"2024-09-11T01:16:35.218868+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:35.218868+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6747,"log_dups_size":0,"ondisk_log_size":6747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3111,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2634},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6579","reported_seq":6664,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.795225+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:34.795225+0000","last_peered":"2024-09-11T01:16:34.795225+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:34.795225+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6579,"log_dups_size":0,"ondisk_log_size":6579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26927587,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4348,"num_write_kb":17392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2548}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6776","reported_seq":6860,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.848908+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:34.848908+0000","last_peered":"2024-09-11T01:16:34.848908+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:34.848908+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4413,"num_write_kb":17652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6726","reported_seq":6844,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.240879+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:35.240879+0000","last_peered":"2024-09-11T01:16:35.240879+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:35.240879+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6726,"log_dups_size":0,"ondisk_log_size":6726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3416,"num_write_kb":13664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2560},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6861","reported_seq":6986,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.247372+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:35.247372+0000","last_peered":"2024-09-11T01:16:35.247372+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:35.247372+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3254,"num_write_kb":13016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6789","reported_seq":6912,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.217885+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:35.217885+0000","last_peered":"2024-09-11T01:16:35.217885+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:35.217885+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27791360,"num_objects":6785,"num_object_clones":0,"num_object_copies":13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3324,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2637},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6669","reported_seq":6794,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.025252+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:35.025252+0000","last_peered":"2024-09-11T01:16:35.025252+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:35.025252+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27268038,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3326,"num_write_kb":13304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440103646,"num_objects":107456,"num_object_clones":0,"num_object_copies":214912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":107456,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":66591,"num_write_kb":266364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1918,"num_bytes_recovered":7856128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":884056064,"data_stored":883900705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107528,"ondisk_log_size":107528,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531500,"kb_used_data":347100,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93840340,"statfs":{"total":96636764160,"available":96092508160,"internally_reserved":0,"allocated":355430400,"data_stored":355173496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266500,"kb_used_data":126516,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94105340,"statfs":{"total":96636764160,"available":96363868160,"internally_reserved":0,"allocated":129552384,"data_stored":129244434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645068,"kb_used_data":388540,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93726772,"statfs":{"total":96636764160,"available":95976214528,"internally_reserved":0,"allocated":397864960,"data_stored":397603979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262663850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,8,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":397967360,"data_stored":397931230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130752512,"data_stored":130669413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355336192,"data_stored":355300062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:36.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:37.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 121 20 108566 108546 3.50339 1.8125 0.0368083 0.0222925 2024-09-11T01:16:37.580 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:37.583 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:37.607 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-11T01:16:36.440204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440103665,"num_objects":107458,"num_object_clones":0,"num_object_copies":214916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":107458,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":66593,"num_write_kb":266366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1919,"num_bytes_recovered":7856147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107531,"ondisk_log_size":107531,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443068,"kb_used_data":862156,"kb_used_omap":12,"kb_used_meta":580851,"kb_avail":281672452,"statfs":{"total":289910292480,"available":288432590848,"internally_reserved":0,"allocated":882847744,"data_stored":882021909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":594791431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,8,14,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-427,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7785,"num_write_kb":31140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003056"},"pg_stats":[{"pgid":"2.f","version":"39'6619","reported_seq":6681,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.262042+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:35.262042+0000","last_peered":"2024-09-11T01:16:35.262042+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:35.262042+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6619,"log_dups_size":0,"ondisk_log_size":6619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":26476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2496},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6735","reported_seq":6805,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.954146+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:34.954146+0000","last_peered":"2024-09-11T01:16:34.954146+0000","last_clean":"2024-09-11T01:16:34.954146+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:34.954146+0000","last_undegraded":"2024-09-11T01:16:34.954146+0000","last_fullsized":"2024-09-11T01:16:34.954146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6735,"log_dups_size":0,"ondisk_log_size":6735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27562950,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6632","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.937059+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:34.937059+0000","last_peered":"2024-09-11T01:16:34.937059+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:34.937059+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4693,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2623}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6670","reported_seq":6705,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.923692+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:34.923692+0000","last_peered":"2024-09-11T01:16:34.923692+0000","last_clean":"2024-09-11T01:16:34.923692+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:34.923692+0000","last_undegraded":"2024-09-11T01:16:34.923692+0000","last_fullsized":"2024-09-11T01:16:34.923692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":26680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6708","reported_seq":6820,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.163390+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:35.163390+0000","last_peered":"2024-09-11T01:16:35.163390+0000","last_clean":"2024-09-11T01:16:35.163390+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:35.163390+0000","last_undegraded":"2024-09-11T01:16:35.163390+0000","last_fullsized":"2024-09-11T01:16:35.163390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6708,"log_dups_size":0,"ondisk_log_size":6708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27447779,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3506,"num_write_kb":14024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6733","reported_seq":6858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.166772+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:35.166772+0000","last_peered":"2024-09-11T01:16:35.166772+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:35.166772+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27558371,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2614},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6869","reported_seq":6993,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.145632+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:35.145632+0000","last_peered":"2024-09-11T01:16:35.145632+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:35.145632+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28111814,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2582},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6711","reported_seq":6821,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.166870+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:35.166870+0000","last_peered":"2024-09-11T01:16:35.166870+0000","last_clean":"2024-09-11T01:16:35.166870+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:35.166870+0000","last_undegraded":"2024-09-11T01:16:35.166870+0000","last_fullsized":"2024-09-11T01:16:35.166870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6711,"log_dups_size":0,"ondisk_log_size":6711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27480064,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6704","reported_seq":10608,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:34.952722+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:34.952722+0000","last_peered":"2024-09-11T01:16:34.952722+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:34.952722+0000","last_undegraded":"2024-09-11T01:16:34.952722+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6704,"log_dups_size":0,"ondisk_log_size":6704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27431395,"num_objects":6698,"num_object_clones":0,"num_object_copies":13396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":6698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4571,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1917,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2207},{"shards":"2","objects":4491}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6747","reported_seq":6865,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.218868+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:35.218868+0000","last_peered":"2024-09-11T01:16:35.218868+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:35.218868+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6747,"log_dups_size":0,"ondisk_log_size":6747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3111,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2634},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6579","reported_seq":6664,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.795225+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:34.795225+0000","last_peered":"2024-09-11T01:16:34.795225+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:34.795225+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6579,"log_dups_size":0,"ondisk_log_size":6579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26927587,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4348,"num_write_kb":17392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2548}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6776","reported_seq":6860,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.848908+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:34.848908+0000","last_peered":"2024-09-11T01:16:34.848908+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:34.848908+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4413,"num_write_kb":17652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6726","reported_seq":6844,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.240879+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:35.240879+0000","last_peered":"2024-09-11T01:16:35.240879+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:35.240879+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6726,"log_dups_size":0,"ondisk_log_size":6726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3416,"num_write_kb":13664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2560},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6861","reported_seq":6986,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.247372+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:35.247372+0000","last_peered":"2024-09-11T01:16:35.247372+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:35.247372+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3254,"num_write_kb":13016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6789","reported_seq":6912,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.217885+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:35.217885+0000","last_peered":"2024-09-11T01:16:35.217885+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:35.217885+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27791360,"num_objects":6785,"num_object_clones":0,"num_object_copies":13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3324,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2637},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6669","reported_seq":6794,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.025252+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:35.025252+0000","last_peered":"2024-09-11T01:16:35.025252+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:35.025252+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27268038,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3326,"num_write_kb":13304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440103646,"num_objects":107456,"num_object_clones":0,"num_object_copies":214912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":107456,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":66591,"num_write_kb":266364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1918,"num_bytes_recovered":7856128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":884056064,"data_stored":883900705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107528,"ondisk_log_size":107528,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531500,"kb_used_data":347100,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93840340,"statfs":{"total":96636764160,"available":96092508160,"internally_reserved":0,"allocated":355430400,"data_stored":355173496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789968,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":266500,"kb_used_data":126516,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94105340,"statfs":{"total":96636764160,"available":96363868160,"internally_reserved":0,"allocated":129552384,"data_stored":129244434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645068,"kb_used_data":388540,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93726772,"statfs":{"total":96636764160,"available":95976214528,"internally_reserved":0,"allocated":397864960,"data_stored":397603979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262663850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,8,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":397967360,"data_stored":397931230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130752512,"data_stored":130669413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355336192,"data_stored":355300062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'6619", 'reported_seq': 6681, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.262042+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:35.262042+0000', 'last_peered': '2024-09-11T01:16:35.262042+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:35.262042+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 6619, 'log_dups_size': 0, 'ondisk_log_size': 6619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27111424, 'num_objects': 6619, 'num_object_clones': 0, 'num_object_copies': 13238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6619, 'num_write_kb': 26476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2496}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'6632", 'reported_seq': 6705, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:34.937059+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:34.937059+0000', 'last_peered': '2024-09-11T01:16:34.937059+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:34.937059+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 6632, 'log_dups_size': 0, 'ondisk_log_size': 6632, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27148288, 'num_objects': 6628, 'num_object_clones': 0, 'num_object_copies': 13256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6628, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4693, 'num_write_kb': 18772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 2623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'6733", 'reported_seq': 6858, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.166772+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:35.166772+0000', 'last_peered': '2024-09-11T01:16:35.166772+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:35.166772+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 6733, 'log_dups_size': 0, 'ondisk_log_size': 6733, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27558371, 'num_objects': 6729, 'num_object_clones': 0, 'num_object_copies': 13458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6729, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3221, 'num_write_kb': 12884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2614}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:37.609 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'6869", 'reported_seq': 6993, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.145632+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:35.145632+0000', 'last_peered': '2024-09-11T01:16:35.145632+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:35.145632+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 6869, 'log_dups_size': 0, 'ondisk_log_size': 6869, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28111814, 'num_objects': 6865, 'num_object_clones': 0, 'num_object_copies': 13730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6865, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3622, 'num_write_kb': 14488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2582}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:37.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'6704", 'reported_seq': 10608, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:34.952722+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:34.952722+0000', 'last_peered': '2024-09-11T01:16:34.952722+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:34.952722+0000', 'last_undegraded': '2024-09-11T01:16:34.952722+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 6704, 'log_dups_size': 0, 'ondisk_log_size': 6704, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27431395, 'num_objects': 6698, 'num_object_clones': 0, 'num_object_copies': 13396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2207, 'num_objects_unfound': 0, 'num_objects_dirty': 6698, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4571, 'num_write_kb': 18284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1917, 'num_bytes_recovered': 7852032, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2207}, {'shards': '2', 'objects': 4491}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:37.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:37.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'6747", 'reported_seq': 6865, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.218868+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:35.218868+0000', 'last_peered': '2024-09-11T01:16:35.218868+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:35.218868+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 6747, 'log_dups_size': 0, 'ondisk_log_size': 6747, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27602944, 'num_objects': 6739, 'num_object_clones': 0, 'num_object_copies': 13478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3111, 'num_write_kb': 12444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2634}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:37.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'6579", 'reported_seq': 6664, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:34.795225+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:34.795225+0000', 'last_peered': '2024-09-11T01:16:34.795225+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:34.795225+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 6579, 'log_dups_size': 0, 'ondisk_log_size': 6579, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26927587, 'num_objects': 6575, 'num_object_clones': 0, 'num_object_copies': 13150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6575, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4348, 'num_write_kb': 17392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 2548}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:37.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:37.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'6776", 'reported_seq': 6860, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:34.848908+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:34.848908+0000', 'last_peered': '2024-09-11T01:16:34.848908+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:34.848908+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 6776, 'log_dups_size': 0, 'ondisk_log_size': 6776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27729920, 'num_objects': 6770, 'num_object_clones': 0, 'num_object_copies': 13540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4413, 'num_write_kb': 17652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 2593}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'6726", 'reported_seq': 6844, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.240879+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:35.240879+0000', 'last_peered': '2024-09-11T01:16:35.240879+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:35.240879+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 6726, 'log_dups_size': 0, 'ondisk_log_size': 6726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27541504, 'num_objects': 6724, 'num_object_clones': 0, 'num_object_copies': 13448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6724, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3416, 'num_write_kb': 13664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2560}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'6861", 'reported_seq': 6986, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.247372+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:35.247372+0000', 'last_peered': '2024-09-11T01:16:35.247372+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:35.247372+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 6861, 'log_dups_size': 0, 'ondisk_log_size': 6861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28069888, 'num_objects': 6853, 'num_object_clones': 0, 'num_object_copies': 13706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3254, 'num_write_kb': 13016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2661}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'6789", 'reported_seq': 6912, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.217885+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:35.217885+0000', 'last_peered': '2024-09-11T01:16:35.217885+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:35.217885+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 6789, 'log_dups_size': 0, 'ondisk_log_size': 6789, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27791360, 'num_objects': 6785, 'num_object_clones': 0, 'num_object_copies': 13570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3324, 'num_write_kb': 13296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2637}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:37.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'6669", 'reported_seq': 6794, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.025252+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:35.025252+0000', 'last_peered': '2024-09-11T01:16:35.025252+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:35.025252+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 6669, 'log_dups_size': 0, 'ondisk_log_size': 6669, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27268038, 'num_objects': 6659, 'num_object_clones': 0, 'num_object_copies': 13318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6659, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3326, 'num_write_kb': 13304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2505}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:37.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:38.336 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:38.336 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/214916 objects degraded (21.175%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":198,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107458,"data_bytes":440103665,"bytes_used":1477701632,"bytes_avail":288432590848,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":214916,"degraded_ratio":0.21174784566993615,"misplaced_objects":2207,"misplaced_total":214916,"misplaced_ratio":0.010269128403655382,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":145712,"recovering_keys_per_sec":0,"num_objects_recovered":427,"num_bytes_recovered":1748992,"num_keys_recovered":0,"write_bytes_sec":2656603,"read_op_per_sec":0,"write_op_per_sec":648},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:38.339 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 122 20 109034 109014 3.48965 1.82812 0.0799795 0.0223791 2024-09-11T01:16:38.365 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:38.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:38.961 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:38.962 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:38.973 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-11T01:16:38.440703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440103665,"num_objects":107458,"num_object_clones":0,"num_object_copies":214916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":107458,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":66593,"num_write_kb":266366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1919,"num_bytes_recovered":7856147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107531,"ondisk_log_size":107531,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1457072,"kb_used_data":872000,"kb_used_omap":12,"kb_used_meta":585011,"kb_avail":281658448,"statfs":{"total":289910292480,"available":288418250752,"internally_reserved":0,"allocated":892928000,"data_stored":892102165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":599051271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,8,14,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21037056,"num_objects":5136,"num_object_clones":0,"num_object_copies":10272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-289,"num_objects_unfound":0,"num_objects_dirty":5136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5136,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":1183744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003109"},"pg_stats":[{"pgid":"2.f","version":"39'6619","reported_seq":6681,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.262042+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:35.262042+0000","last_peered":"2024-09-11T01:16:35.262042+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:35.262042+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6619,"log_dups_size":0,"ondisk_log_size":6619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6619,"num_write_kb":26476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2496},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6735","reported_seq":6805,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.954146+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:34.954146+0000","last_peered":"2024-09-11T01:16:34.954146+0000","last_clean":"2024-09-11T01:16:34.954146+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:34.954146+0000","last_undegraded":"2024-09-11T01:16:34.954146+0000","last_fullsized":"2024-09-11T01:16:34.954146+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6735,"log_dups_size":0,"ondisk_log_size":6735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27562950,"num_objects":6731,"num_object_clones":0,"num_object_copies":13462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6731,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4833,"num_write_kb":19332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6632","reported_seq":6705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.937059+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:34.937059+0000","last_peered":"2024-09-11T01:16:34.937059+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:34.937059+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6632,"log_dups_size":0,"ondisk_log_size":6632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27148288,"num_objects":6628,"num_object_clones":0,"num_object_copies":13256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4693,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2623}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6670","reported_seq":6705,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.923692+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:34.923692+0000","last_peered":"2024-09-11T01:16:34.923692+0000","last_clean":"2024-09-11T01:16:34.923692+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:34.923692+0000","last_undegraded":"2024-09-11T01:16:34.923692+0000","last_fullsized":"2024-09-11T01:16:34.923692+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320320,"num_objects":6670,"num_object_clones":0,"num_object_copies":13340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6670,"num_write_kb":26680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6708","reported_seq":6820,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.163390+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:35.163390+0000","last_peered":"2024-09-11T01:16:35.163390+0000","last_clean":"2024-09-11T01:16:35.163390+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:35.163390+0000","last_undegraded":"2024-09-11T01:16:35.163390+0000","last_fullsized":"2024-09-11T01:16:35.163390+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6708,"log_dups_size":0,"ondisk_log_size":6708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27447779,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3506,"num_write_kb":14024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6733","reported_seq":6858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.166772+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:35.166772+0000","last_peered":"2024-09-11T01:16:35.166772+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:35.166772+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27558371,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2614},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'6869","reported_seq":6993,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.145632+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:35.145632+0000","last_peered":"2024-09-11T01:16:35.145632+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:35.145632+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":6869,"log_dups_size":0,"ondisk_log_size":6869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28111814,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3622,"num_write_kb":14488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2582},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6711","reported_seq":6821,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.166870+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:35.166870+0000","last_peered":"2024-09-11T01:16:35.166870+0000","last_clean":"2024-09-11T01:16:35.166870+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:35.166870+0000","last_undegraded":"2024-09-11T01:16:35.166870+0000","last_fullsized":"2024-09-11T01:16:35.166870+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6711,"log_dups_size":0,"ondisk_log_size":6711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27480064,"num_objects":6709,"num_object_clones":0,"num_object_copies":13418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6704","reported_seq":10608,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:34.952722+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:34.952722+0000","last_peered":"2024-09-11T01:16:34.952722+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:34.952722+0000","last_undegraded":"2024-09-11T01:16:34.952722+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6704,"log_dups_size":0,"ondisk_log_size":6704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27431395,"num_objects":6698,"num_object_clones":0,"num_object_copies":13396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":6698,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4571,"num_write_kb":18284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1917,"num_bytes_recovered":7852032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2207},{"shards":"2","objects":4491}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6747","reported_seq":6865,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.218868+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:35.218868+0000","last_peered":"2024-09-11T01:16:35.218868+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:35.218868+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6747,"log_dups_size":0,"ondisk_log_size":6747,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3111,"num_write_kb":12444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2634},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6579","reported_seq":6664,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.795225+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:34.795225+0000","last_peered":"2024-09-11T01:16:34.795225+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:34.795225+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6579,"log_dups_size":0,"ondisk_log_size":6579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26927587,"num_objects":6575,"num_object_clones":0,"num_object_copies":13150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6575,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4348,"num_write_kb":17392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2548}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6776","reported_seq":6860,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.848908+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:34.848908+0000","last_peered":"2024-09-11T01:16:34.848908+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:34.848908+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4413,"num_write_kb":17652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2593}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6726","reported_seq":6844,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.240879+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:35.240879+0000","last_peered":"2024-09-11T01:16:35.240879+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:35.240879+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6726,"log_dups_size":0,"ondisk_log_size":6726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541504,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3416,"num_write_kb":13664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2560},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6861","reported_seq":6986,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.247372+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:35.247372+0000","last_peered":"2024-09-11T01:16:35.247372+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:35.247372+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6861,"log_dups_size":0,"ondisk_log_size":6861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28069888,"num_objects":6853,"num_object_clones":0,"num_object_copies":13706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3254,"num_write_kb":13016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2661},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6789","reported_seq":6912,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.217885+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:35.217885+0000","last_peered":"2024-09-11T01:16:35.217885+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:35.217885+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6789,"log_dups_size":0,"ondisk_log_size":6789,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27791360,"num_objects":6785,"num_object_clones":0,"num_object_copies":13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3324,"num_write_kb":13296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2637},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6669","reported_seq":6794,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.025252+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:35.025252+0000","last_peered":"2024-09-11T01:16:35.025252+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:35.025252+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6669,"log_dups_size":0,"ondisk_log_size":6669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27268038,"num_objects":6659,"num_object_clones":0,"num_object_copies":13318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6659,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3326,"num_write_kb":13304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440103646,"num_objects":107456,"num_object_clones":0,"num_object_copies":214912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2207,"num_objects_unfound":0,"num_objects_dirty":107456,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":66591,"num_write_kb":266364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1918,"num_bytes_recovered":7856128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":892698624,"data_stored":892543265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107528,"ondisk_log_size":107528,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531500,"kb_used_data":347100,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93840340,"statfs":{"total":96636764160,"available":96092508160,"internally_reserved":0,"allocated":355430400,"data_stored":355173496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":188804782},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789969,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":280504,"kb_used_data":136360,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94091336,"statfs":{"total":96636764160,"available":96349528064,"internally_reserved":0,"allocated":139632640,"data_stored":139324690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":645068,"kb_used_data":388540,"kb_used_omap":4,"kb_used_meta":256507,"kb_avail":93726772,"statfs":{"total":96636764160,"available":95976214528,"internally_reserved":0,"allocated":397864960,"data_stored":397603979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":262663850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,8,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":397967360,"data_stored":397931230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139395072,"data_stored":139311973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355336192,"data_stored":355300062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:39.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 123 20 109469 109449 3.4751 1.69922 0.0252714 0.0224733 2024-09-11T01:16:40.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 124 20 110006 109986 3.46399 2.09766 0.0463645 0.0225458 2024-09-11T01:16:41.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 125 20 110540 110520 3.45297 2.08594 0.0273136 0.0226178 2024-09-11T01:16:41.975 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:42.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 126 20 111049 111029 3.44134 1.98828 0.0677622 0.0226935 2024-09-11T01:16:42.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:42.545 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:42.575 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-11T01:16:42.443478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":450155249,"num_objects":109912,"num_object_clones":0,"num_object_copies":219824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":109912,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":69047,"num_write_kb":276182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2073,"num_bytes_recovered":8479222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109985,"ondisk_log_size":109985,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462264,"kb_used_data":884104,"kb_used_omap":12,"kb_used_meta":578099,"kb_avail":281653256,"statfs":{"total":289910292480,"available":288412934144,"internally_reserved":0,"allocated":905322496,"data_stored":904496661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":591973383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,16,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20553728,"num_objects":5018,"num_object_clones":0,"num_object_copies":10036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":5018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1221091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004785"},"pg_stats":[{"pgid":"2.f","version":"39'6772","reported_seq":6836,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.185808+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:40.185808+0000","last_peered":"2024-09-11T01:16:40.185808+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:40.185808+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6772,"log_dups_size":0,"ondisk_log_size":6772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6772,"num_write_kb":27088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2649},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6886","reported_seq":6957,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.939139+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:39.939139+0000","last_peered":"2024-09-11T01:16:39.939139+0000","last_clean":"2024-09-11T01:16:39.939139+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:39.939139+0000","last_undegraded":"2024-09-11T01:16:39.939139+0000","last_fullsized":"2024-09-11T01:16:39.939139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6886,"log_dups_size":0,"ondisk_log_size":6886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28181446,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4984,"num_write_kb":19936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6774","reported_seq":6847,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.945717+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:39.945717+0000","last_peered":"2024-09-11T01:16:39.945717+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:39.945717+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6774,"log_dups_size":0,"ondisk_log_size":6774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6811","reported_seq":6845,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.936048+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:39.936048+0000","last_peered":"2024-09-11T01:16:39.936048+0000","last_clean":"2024-09-11T01:16:39.936048+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:39.936048+0000","last_undegraded":"2024-09-11T01:16:39.936048+0000","last_fullsized":"2024-09-11T01:16:39.936048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6811,"log_dups_size":0,"ondisk_log_size":6811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6866","reported_seq":6978,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.295673+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:40.295673+0000","last_peered":"2024-09-11T01:16:40.295673+0000","last_clean":"2024-09-11T01:16:40.295673+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:40.295673+0000","last_undegraded":"2024-09-11T01:16:40.295673+0000","last_fullsized":"2024-09-11T01:16:40.295673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6866,"log_dups_size":0,"ondisk_log_size":6866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094947,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6870","reported_seq":6995,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.179428+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:40.179428+0000","last_peered":"2024-09-11T01:16:40.179428+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:40.179428+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6870,"log_dups_size":0,"ondisk_log_size":6870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119523,"num_objects":6866,"num_object_clones":0,"num_object_copies":13732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3358,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2751},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7046","reported_seq":7170,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.254691+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:40.254691+0000","last_peered":"2024-09-11T01:16:40.254691+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:40.254691+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28836806,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3799,"num_write_kb":15196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2759},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6877","reported_seq":6987,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.251823+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:40.251823+0000","last_peered":"2024-09-11T01:16:40.251823+0000","last_clean":"2024-09-11T01:16:40.251823+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:40.251823+0000","last_undegraded":"2024-09-11T01:16:40.251823+0000","last_fullsized":"2024-09-11T01:16:40.251823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3830,"num_write_kb":15320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6848","reported_seq":11064,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:39.956398+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:39.956398+0000","last_peered":"2024-09-11T01:16:39.956398+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:39.956398+0000","last_undegraded":"2024-09-11T01:16:39.956398+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28021219,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2071,"num_bytes_recovered":8475107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2054},{"shards":"2","objects":4788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6914","reported_seq":7033,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.294148+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:40.294148+0000","last_peered":"2024-09-11T01:16:40.294148+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:40.294148+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6914,"log_dups_size":0,"ondisk_log_size":6914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28286976,"num_objects":6906,"num_object_clones":0,"num_object_copies":13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":13112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2801},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6733","reported_seq":6818,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.905442+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:39.905442+0000","last_peered":"2024-09-11T01:16:39.905442+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:39.905442+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27558371,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4502,"num_write_kb":18008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6953","reported_seq":7037,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.952057+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:39.952057+0000","last_peered":"2024-09-11T01:16:39.952057+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:39.952057+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":18360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2770}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6872","reported_seq":6992,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.274931+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:40.274931+0000","last_peered":"2024-09-11T01:16:40.274931+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:40.274931+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6872,"log_dups_size":0,"ondisk_log_size":6872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28139520,"num_objects":6870,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3562,"num_write_kb":14248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2706},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6997","reported_seq":7122,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.196971+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:40.196971+0000","last_peered":"2024-09-11T01:16:40.196971+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:40.196971+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28626944,"num_objects":6989,"num_object_clones":0,"num_object_copies":13978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3390,"num_write_kb":13560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6949","reported_seq":7073,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.170168+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:40.170168+0000","last_peered":"2024-09-11T01:16:40.170168+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:40.170168+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6949,"log_dups_size":0,"ondisk_log_size":6949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3484,"num_write_kb":13936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6814","reported_seq":6939,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.288837+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:40.288837+0000","last_peered":"2024-09-11T01:16:40.288837+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:40.288837+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27861958,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3471,"num_write_kb":13884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2650},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":450155230,"num_objects":109910,"num_object_clones":0,"num_object_copies":219820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":109910,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":69045,"num_write_kb":276180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2072,"num_bytes_recovered":8479203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":905232384,"data_stored":905077025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109982,"ondisk_log_size":109982,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523872,"kb_used_data":351952,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93847968,"statfs":{"total":96636764160,"available":96100319232,"internally_reserved":0,"allocated":360398848,"data_stored":360141944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789969,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":280504,"kb_used_data":136360,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94091336,"statfs":{"total":96636764160,"available":96349528064,"internally_reserved":0,"allocated":139632640,"data_stored":139324690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657888,"kb_used_data":395792,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93713952,"statfs":{"total":96636764160,"available":95963086848,"internally_reserved":0,"allocated":405291008,"data_stored":405030027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405487616,"data_stored":405451486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139395072,"data_stored":139311973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360349696,"data_stored":360313566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:42.577 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:43.127 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:43.129 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:43.150 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-11T01:16:42.443478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":450155249,"num_objects":109912,"num_object_clones":0,"num_object_copies":219824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":109912,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":69047,"num_write_kb":276182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2073,"num_bytes_recovered":8479222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109985,"ondisk_log_size":109985,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462264,"kb_used_data":884104,"kb_used_omap":12,"kb_used_meta":578099,"kb_avail":281653256,"statfs":{"total":289910292480,"available":288412934144,"internally_reserved":0,"allocated":905322496,"data_stored":904496661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":591973383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,16,8,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20553728,"num_objects":5018,"num_object_clones":0,"num_object_copies":10036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":5018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1221091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004785"},"pg_stats":[{"pgid":"2.f","version":"39'6772","reported_seq":6836,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.185808+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:40.185808+0000","last_peered":"2024-09-11T01:16:40.185808+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:40.185808+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6772,"log_dups_size":0,"ondisk_log_size":6772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6772,"num_write_kb":27088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2649},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6886","reported_seq":6957,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.939139+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:39.939139+0000","last_peered":"2024-09-11T01:16:39.939139+0000","last_clean":"2024-09-11T01:16:39.939139+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:39.939139+0000","last_undegraded":"2024-09-11T01:16:39.939139+0000","last_fullsized":"2024-09-11T01:16:39.939139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6886,"log_dups_size":0,"ondisk_log_size":6886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28181446,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4984,"num_write_kb":19936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6774","reported_seq":6847,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.945717+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:39.945717+0000","last_peered":"2024-09-11T01:16:39.945717+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:39.945717+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6774,"log_dups_size":0,"ondisk_log_size":6774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6811","reported_seq":6845,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.936048+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:39.936048+0000","last_peered":"2024-09-11T01:16:39.936048+0000","last_clean":"2024-09-11T01:16:39.936048+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:39.936048+0000","last_undegraded":"2024-09-11T01:16:39.936048+0000","last_fullsized":"2024-09-11T01:16:39.936048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6811,"log_dups_size":0,"ondisk_log_size":6811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6866","reported_seq":6978,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.295673+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:40.295673+0000","last_peered":"2024-09-11T01:16:40.295673+0000","last_clean":"2024-09-11T01:16:40.295673+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:40.295673+0000","last_undegraded":"2024-09-11T01:16:40.295673+0000","last_fullsized":"2024-09-11T01:16:40.295673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6866,"log_dups_size":0,"ondisk_log_size":6866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094947,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6870","reported_seq":6995,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.179428+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:40.179428+0000","last_peered":"2024-09-11T01:16:40.179428+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:40.179428+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6870,"log_dups_size":0,"ondisk_log_size":6870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119523,"num_objects":6866,"num_object_clones":0,"num_object_copies":13732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3358,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2751},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7046","reported_seq":7170,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.254691+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:40.254691+0000","last_peered":"2024-09-11T01:16:40.254691+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:40.254691+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28836806,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3799,"num_write_kb":15196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2759},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6877","reported_seq":6987,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.251823+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:40.251823+0000","last_peered":"2024-09-11T01:16:40.251823+0000","last_clean":"2024-09-11T01:16:40.251823+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:40.251823+0000","last_undegraded":"2024-09-11T01:16:40.251823+0000","last_fullsized":"2024-09-11T01:16:40.251823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3830,"num_write_kb":15320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6848","reported_seq":11064,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:39.956398+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:39.956398+0000","last_peered":"2024-09-11T01:16:39.956398+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:39.956398+0000","last_undegraded":"2024-09-11T01:16:39.956398+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28021219,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2071,"num_bytes_recovered":8475107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2054},{"shards":"2","objects":4788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6914","reported_seq":7033,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.294148+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:40.294148+0000","last_peered":"2024-09-11T01:16:40.294148+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:40.294148+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6914,"log_dups_size":0,"ondisk_log_size":6914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28286976,"num_objects":6906,"num_object_clones":0,"num_object_copies":13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":13112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2801},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6733","reported_seq":6818,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.905442+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:39.905442+0000","last_peered":"2024-09-11T01:16:39.905442+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:39.905442+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27558371,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4502,"num_write_kb":18008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6953","reported_seq":7037,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.952057+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:39.952057+0000","last_peered":"2024-09-11T01:16:39.952057+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:39.952057+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":18360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2770}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6872","reported_seq":6992,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.274931+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:40.274931+0000","last_peered":"2024-09-11T01:16:40.274931+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:40.274931+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6872,"log_dups_size":0,"ondisk_log_size":6872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28139520,"num_objects":6870,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3562,"num_write_kb":14248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2706},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6997","reported_seq":7122,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.196971+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:40.196971+0000","last_peered":"2024-09-11T01:16:40.196971+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:40.196971+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28626944,"num_objects":6989,"num_object_clones":0,"num_object_copies":13978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3390,"num_write_kb":13560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6949","reported_seq":7073,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.170168+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:40.170168+0000","last_peered":"2024-09-11T01:16:40.170168+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:40.170168+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6949,"log_dups_size":0,"ondisk_log_size":6949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3484,"num_write_kb":13936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6814","reported_seq":6939,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.288837+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:40.288837+0000","last_peered":"2024-09-11T01:16:40.288837+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:40.288837+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27861958,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3471,"num_write_kb":13884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2650},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":450155230,"num_objects":109910,"num_object_clones":0,"num_object_copies":219820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":109910,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":69045,"num_write_kb":276180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2072,"num_bytes_recovered":8479203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":905232384,"data_stored":905077025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109982,"ondisk_log_size":109982,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523872,"kb_used_data":351952,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93847968,"statfs":{"total":96636764160,"available":96100319232,"internally_reserved":0,"allocated":360398848,"data_stored":360141944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789969,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":280504,"kb_used_data":136360,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":94091336,"statfs":{"total":96636764160,"available":96349528064,"internally_reserved":0,"allocated":139632640,"data_stored":139324690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147582639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657888,"kb_used_data":395792,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93713952,"statfs":{"total":96636764160,"available":95963086848,"internally_reserved":0,"allocated":405291008,"data_stored":405030027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405487616,"data_stored":405451486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139395072,"data_stored":139311973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360349696,"data_stored":360313566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'6772", 'reported_seq': 6836, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.185808+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:40.185808+0000', 'last_peered': '2024-09-11T01:16:40.185808+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:40.185808+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 6772, 'log_dups_size': 0, 'ondisk_log_size': 6772, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27738112, 'num_objects': 6772, 'num_object_clones': 0, 'num_object_copies': 13544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6772, 'num_write_kb': 27088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2649}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'6774", 'reported_seq': 6847, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:39.945717+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:39.945717+0000', 'last_peered': '2024-09-11T01:16:39.945717+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:39.945717+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 6774, 'log_dups_size': 0, 'ondisk_log_size': 6774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27729920, 'num_objects': 6770, 'num_object_clones': 0, 'num_object_copies': 13540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4835, 'num_write_kb': 19340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 2765}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'6870", 'reported_seq': 6995, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.179428+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:40.179428+0000', 'last_peered': '2024-09-11T01:16:40.179428+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:40.179428+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 6870, 'log_dups_size': 0, 'ondisk_log_size': 6870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28119523, 'num_objects': 6866, 'num_object_clones': 0, 'num_object_copies': 13732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6866, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3358, 'num_write_kb': 13432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2751}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'7046", 'reported_seq': 7170, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.254691+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:40.254691+0000', 'last_peered': '2024-09-11T01:16:40.254691+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:40.254691+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 7046, 'log_dups_size': 0, 'ondisk_log_size': 7046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28836806, 'num_objects': 7042, 'num_object_clones': 0, 'num_object_copies': 14084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7042, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3799, 'num_write_kb': 15196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2759}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:43.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'6848", 'reported_seq': 11064, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:39.956398+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:39.956398+0000', 'last_peered': '2024-09-11T01:16:39.956398+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:39.956398+0000', 'last_undegraded': '2024-09-11T01:16:39.956398+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 6848, 'log_dups_size': 0, 'ondisk_log_size': 6848, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28021219, 'num_objects': 6842, 'num_object_clones': 0, 'num_object_copies': 13684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2054, 'num_objects_unfound': 0, 'num_objects_dirty': 6842, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4715, 'num_write_kb': 18860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2071, 'num_bytes_recovered': 8475107, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2054}, {'shards': '2', 'objects': 4788}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'6914", 'reported_seq': 7033, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.294148+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:40.294148+0000', 'last_peered': '2024-09-11T01:16:40.294148+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:40.294148+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 6914, 'log_dups_size': 0, 'ondisk_log_size': 6914, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28286976, 'num_objects': 6906, 'num_object_clones': 0, 'num_object_copies': 13812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3278, 'num_write_kb': 13112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2801}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'6733", 'reported_seq': 6818, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:39.905442+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:39.905442+0000', 'last_peered': '2024-09-11T01:16:39.905442+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:39.905442+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 6733, 'log_dups_size': 0, 'ondisk_log_size': 6733, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27558371, 'num_objects': 6729, 'num_object_clones': 0, 'num_object_copies': 13458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6729, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4502, 'num_write_kb': 18008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 2702}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'6953", 'reported_seq': 7037, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:39.952057+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:39.952057+0000', 'last_peered': '2024-09-11T01:16:39.952057+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:39.952057+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 6953, 'log_dups_size': 0, 'ondisk_log_size': 6953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28454912, 'num_objects': 6947, 'num_object_clones': 0, 'num_object_copies': 13894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4590, 'num_write_kb': 18360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 2770}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'6872", 'reported_seq': 6992, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.274931+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:40.274931+0000', 'last_peered': '2024-09-11T01:16:40.274931+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:40.274931+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 6872, 'log_dups_size': 0, 'ondisk_log_size': 6872, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28139520, 'num_objects': 6870, 'num_object_clones': 0, 'num_object_copies': 13740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3562, 'num_write_kb': 14248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2706}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'6997", 'reported_seq': 7122, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.196971+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:40.196971+0000', 'last_peered': '2024-09-11T01:16:40.196971+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:40.196971+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 6997, 'log_dups_size': 0, 'ondisk_log_size': 6997, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28626944, 'num_objects': 6989, 'num_object_clones': 0, 'num_object_copies': 13978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3390, 'num_write_kb': 13560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2797}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.153 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:43.154 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'6949", 'reported_seq': 7073, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.170168+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:40.170168+0000', 'last_peered': '2024-09-11T01:16:40.170168+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:40.170168+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 6949, 'log_dups_size': 0, 'ondisk_log_size': 6949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28446720, 'num_objects': 6945, 'num_object_clones': 0, 'num_object_copies': 13890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3484, 'num_write_kb': 13936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2797}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.154 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:43.154 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'6814", 'reported_seq': 6939, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.288837+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:40.288837+0000', 'last_peered': '2024-09-11T01:16:40.288837+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:40.288837+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 6814, 'log_dups_size': 0, 'ondisk_log_size': 6814, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27861958, 'num_objects': 6804, 'num_object_clones': 0, 'num_object_copies': 13608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6804, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3471, 'num_write_kb': 13884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2650}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:43.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:43.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 127 20 111513 111493 3.42852 1.8125 0.0521319 0.0227769 2024-09-11T01:16:43.876 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:43.876 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/219824 objects degraded (20.702%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":204,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109912,"data_bytes":450155249,"bytes_used":1497358336,"bytes_avail":288412934144,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":219824,"degraded_ratio":0.20702016158381251,"misplaced_objects":2054,"misplaced_total":219824,"misplaced_ratio":0.0093438387073295007,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":101717,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":1221091,"num_keys_recovered":0,"write_bytes_sec":1712127,"read_op_per_sec":0,"write_op_per_sec":417},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:43.902 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:43.902 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:44.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 128 20 111985 111965 3.41614 1.84375 0.0538817 0.0228589 2024-09-11T01:16:44.474 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:44.477 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:44.500 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-11T01:16:44.443975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":450155249,"num_objects":109912,"num_object_clones":0,"num_object_copies":219824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":109912,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":69047,"num_write_kb":276182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2073,"num_bytes_recovered":8479222,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109985,"ondisk_log_size":109985,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496792,"kb_used_data":892260,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281618728,"statfs":{"total":289910292480,"available":288377577472,"internally_reserved":0,"allocated":913674240,"data_stored":912852018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":618974215},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,3,13,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20553728,"num_objects":5018,"num_object_clones":0,"num_object_copies":10036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-299,"num_objects_unfound":0,"num_objects_dirty":5018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1221091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004738"},"pg_stats":[{"pgid":"2.f","version":"39'6772","reported_seq":6836,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.185808+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:40.185808+0000","last_peered":"2024-09-11T01:16:40.185808+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:40.185808+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6772,"log_dups_size":0,"ondisk_log_size":6772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27738112,"num_objects":6772,"num_object_clones":0,"num_object_copies":13544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6772,"num_write_kb":27088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2649},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'6886","reported_seq":6957,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.939139+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:39.939139+0000","last_peered":"2024-09-11T01:16:39.939139+0000","last_clean":"2024-09-11T01:16:39.939139+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:39.939139+0000","last_undegraded":"2024-09-11T01:16:39.939139+0000","last_fullsized":"2024-09-11T01:16:39.939139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":6886,"log_dups_size":0,"ondisk_log_size":6886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28181446,"num_objects":6882,"num_object_clones":0,"num_object_copies":13764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6882,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4984,"num_write_kb":19936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6774","reported_seq":6847,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.945717+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:39.945717+0000","last_peered":"2024-09-11T01:16:39.945717+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:39.945717+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6774,"log_dups_size":0,"ondisk_log_size":6774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6811","reported_seq":6845,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.936048+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:39.936048+0000","last_peered":"2024-09-11T01:16:39.936048+0000","last_clean":"2024-09-11T01:16:39.936048+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:39.936048+0000","last_undegraded":"2024-09-11T01:16:39.936048+0000","last_fullsized":"2024-09-11T01:16:39.936048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6811,"log_dups_size":0,"ondisk_log_size":6811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6811,"num_write_kb":27244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'6866","reported_seq":6978,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.295673+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:40.295673+0000","last_peered":"2024-09-11T01:16:40.295673+0000","last_clean":"2024-09-11T01:16:40.295673+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:40.295673+0000","last_undegraded":"2024-09-11T01:16:40.295673+0000","last_fullsized":"2024-09-11T01:16:40.295673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":6866,"log_dups_size":0,"ondisk_log_size":6866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28094947,"num_objects":6860,"num_object_clones":0,"num_object_copies":13720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3664,"num_write_kb":14656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'6870","reported_seq":6995,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.179428+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:40.179428+0000","last_peered":"2024-09-11T01:16:40.179428+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:40.179428+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":6870,"log_dups_size":0,"ondisk_log_size":6870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119523,"num_objects":6866,"num_object_clones":0,"num_object_copies":13732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6866,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3358,"num_write_kb":13432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2751},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7046","reported_seq":7170,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.254691+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:40.254691+0000","last_peered":"2024-09-11T01:16:40.254691+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:40.254691+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28836806,"num_objects":7042,"num_object_clones":0,"num_object_copies":14084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7042,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3799,"num_write_kb":15196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2759},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'6877","reported_seq":6987,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.251823+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:40.251823+0000","last_peered":"2024-09-11T01:16:40.251823+0000","last_clean":"2024-09-11T01:16:40.251823+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:40.251823+0000","last_undegraded":"2024-09-11T01:16:40.251823+0000","last_fullsized":"2024-09-11T01:16:40.251823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":6877,"log_dups_size":0,"ondisk_log_size":6877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28160000,"num_objects":6875,"num_object_clones":0,"num_object_copies":13750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3830,"num_write_kb":15320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6848","reported_seq":11064,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:39.956398+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:39.956398+0000","last_peered":"2024-09-11T01:16:39.956398+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:39.956398+0000","last_undegraded":"2024-09-11T01:16:39.956398+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6848,"log_dups_size":0,"ondisk_log_size":6848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28021219,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4715,"num_write_kb":18860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2071,"num_bytes_recovered":8475107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2054},{"shards":"2","objects":4788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'6914","reported_seq":7033,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.294148+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:40.294148+0000","last_peered":"2024-09-11T01:16:40.294148+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:40.294148+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":6914,"log_dups_size":0,"ondisk_log_size":6914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28286976,"num_objects":6906,"num_object_clones":0,"num_object_copies":13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3278,"num_write_kb":13112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2801},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6733","reported_seq":6818,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.905442+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:39.905442+0000","last_peered":"2024-09-11T01:16:39.905442+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:39.905442+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6733,"log_dups_size":0,"ondisk_log_size":6733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27558371,"num_objects":6729,"num_object_clones":0,"num_object_copies":13458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4502,"num_write_kb":18008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'6953","reported_seq":7037,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.952057+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:39.952057+0000","last_peered":"2024-09-11T01:16:39.952057+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:39.952057+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4590,"num_write_kb":18360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2770}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'6872","reported_seq":6992,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.274931+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:40.274931+0000","last_peered":"2024-09-11T01:16:40.274931+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:40.274931+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":6872,"log_dups_size":0,"ondisk_log_size":6872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28139520,"num_objects":6870,"num_object_clones":0,"num_object_copies":13740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3562,"num_write_kb":14248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2706},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'6997","reported_seq":7122,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.196971+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:40.196971+0000","last_peered":"2024-09-11T01:16:40.196971+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:40.196971+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28626944,"num_objects":6989,"num_object_clones":0,"num_object_copies":13978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3390,"num_write_kb":13560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'6949","reported_seq":7073,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.170168+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:40.170168+0000","last_peered":"2024-09-11T01:16:40.170168+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:40.170168+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":6949,"log_dups_size":0,"ondisk_log_size":6949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3484,"num_write_kb":13936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2797},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6814","reported_seq":6939,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.288837+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:40.288837+0000","last_peered":"2024-09-11T01:16:40.288837+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:40.288837+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27861958,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3471,"num_write_kb":13884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2650},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":450155230,"num_objects":109910,"num_object_clones":0,"num_object_copies":219820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":2054,"num_objects_unfound":0,"num_objects_dirty":109910,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":69045,"num_write_kb":276180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2072,"num_bytes_recovered":8479203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":913522688,"data_stored":913370942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109982,"ondisk_log_size":109982,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523872,"kb_used_data":351952,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93847968,"statfs":{"total":96636764160,"available":96100319232,"internally_reserved":0,"allocated":360398848,"data_stored":360141944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":176025262},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315032,"kb_used_data":144516,"kb_used_omap":4,"kb_used_meta":170491,"kb_avail":94056808,"statfs":{"total":96636764160,"available":96314171392,"internally_reserved":0,"allocated":147984384,"data_stored":147680047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":174583471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657888,"kb_used_data":395792,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93713952,"statfs":{"total":96636764160,"available":95963086848,"internally_reserved":0,"allocated":405291008,"data_stored":405030027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":405487616,"data_stored":405451486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147685376,"data_stored":147605890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360349696,"data_stored":360313566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:45.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 129 20 112505 112485 3.4054 2.03125 0.022814 0.0229339 2024-09-11T01:16:46.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 130 20 113042 113022 3.39534 2.09766 0.0568847 0.0230017 2024-09-11T01:16:47.340 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 131 20 113562 113542 3.38493 2.03125 0.0273362 0.0230731 2024-09-11T01:16:47.673 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:48.236 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:48.237 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:48.267 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-11T01:16:46.446089+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460505841,"num_objects":112439,"num_object_clones":0,"num_object_copies":224878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":112439,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":71574,"num_write_kb":286290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2223,"num_bytes_recovered":9093622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112512,"ondisk_log_size":112512,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517664,"kb_used_data":904812,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281597856,"statfs":{"total":289910292480,"available":288356204544,"internally_reserved":0,"allocated":926527488,"data_stored":925705266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":627493895},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,9,5,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30904320,"num_objects":7545,"num_object_clones":0,"num_object_copies":15090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-449,"num_objects_unfound":0,"num_objects_dirty":7545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7545,"num_write_kb":30180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":1835491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006343"},"pg_stats":[{"pgid":"2.f","version":"39'6936","reported_seq":7000,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.331548+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:45.331548+0000","last_peered":"2024-09-11T01:16:45.331548+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:45.331548+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6936,"num_write_kb":27744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2813},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7038","reported_seq":7109,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.893260+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:44.893260+0000","last_peered":"2024-09-11T01:16:44.893260+0000","last_clean":"2024-09-11T01:16:44.893260+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:44.893260+0000","last_undegraded":"2024-09-11T01:16:44.893260+0000","last_fullsized":"2024-09-11T01:16:44.893260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7038,"log_dups_size":0,"ondisk_log_size":7038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28804038,"num_objects":7034,"num_object_clones":0,"num_object_copies":14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5136,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6936","reported_seq":7009,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.890655+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:44.890655+0000","last_peered":"2024-09-11T01:16:44.890655+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:44.890655+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28393472,"num_objects":6932,"num_object_clones":0,"num_object_copies":13864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6985","reported_seq":7021,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.913374+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:44.913374+0000","last_peered":"2024-09-11T01:16:44.913374+0000","last_clean":"2024-09-11T01:16:44.913374+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:44.913374+0000","last_undegraded":"2024-09-11T01:16:44.913374+0000","last_fullsized":"2024-09-11T01:16:44.913374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6985,"log_dups_size":0,"ondisk_log_size":6985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7007","reported_seq":7119,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.295461+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:45.295461+0000","last_peered":"2024-09-11T01:16:45.295461+0000","last_clean":"2024-09-11T01:16:45.295461+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:45.295461+0000","last_undegraded":"2024-09-11T01:16:45.295461+0000","last_fullsized":"2024-09-11T01:16:45.295461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7007,"log_dups_size":0,"ondisk_log_size":7007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672483,"num_objects":7001,"num_object_clones":0,"num_object_copies":14002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3805,"num_write_kb":15220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7026","reported_seq":7151,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.245095+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:45.245095+0000","last_peered":"2024-09-11T01:16:45.245095+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:45.245095+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7026,"log_dups_size":0,"ondisk_log_size":7026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758499,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3514,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2907},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7195","reported_seq":7319,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.316711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:45.316711+0000","last_peered":"2024-09-11T01:16:45.316711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:45.316711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7195,"log_dups_size":0,"ondisk_log_size":7195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29447110,"num_objects":7191,"num_object_clones":0,"num_object_copies":14382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7191,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7052","reported_seq":7162,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.362986+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:45.362986+0000","last_peered":"2024-09-11T01:16:45.362986+0000","last_clean":"2024-09-11T01:16:45.362986+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:45.362986+0000","last_undegraded":"2024-09-11T01:16:45.362986+0000","last_fullsized":"2024-09-11T01:16:45.362986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28876800,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4005,"num_write_kb":16020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6997","reported_seq":11513,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:44.977911+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:44.977911+0000","last_peered":"2024-09-11T01:16:44.977911+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:44.977911+0000","last_undegraded":"2024-09-11T01:16:44.977911+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28631523,"num_objects":6991,"num_object_clones":0,"num_object_copies":13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":6991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4864,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2221,"num_bytes_recovered":9089507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1904},{"shards":"2","objects":5087}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7072","reported_seq":7191,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.311555+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:45.311555+0000","last_peered":"2024-09-11T01:16:45.311555+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:45.311555+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7072,"log_dups_size":0,"ondisk_log_size":7072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28934144,"num_objects":7064,"num_object_clones":0,"num_object_copies":14128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6909","reported_seq":6994,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.958566+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:44.958566+0000","last_peered":"2024-09-11T01:16:44.958566+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:44.958566+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6909,"log_dups_size":0,"ondisk_log_size":6909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28279267,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4678,"num_write_kb":18712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7118","reported_seq":7202,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.943586+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:44.943586+0000","last_peered":"2024-09-11T01:16:44.943586+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:44.943586+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":19020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7030","reported_seq":7150,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.348326+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:45.348326+0000","last_peered":"2024-09-11T01:16:45.348326+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:45.348326+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28786688,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3720,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7155","reported_seq":7280,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.314182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:45.314182+0000","last_peered":"2024-09-11T01:16:45.314182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:45.314182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7155,"log_dups_size":0,"ondisk_log_size":7155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3548,"num_write_kb":14192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2955},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7096","reported_seq":7218,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.351339+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:45.351339+0000","last_peered":"2024-09-11T01:16:45.351339+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:45.351339+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7096,"log_dups_size":0,"ondisk_log_size":7096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29048832,"num_objects":7092,"num_object_clones":0,"num_object_copies":14184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2944},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6957","reported_seq":7080,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.339700+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:45.339700+0000","last_peered":"2024-09-11T01:16:45.339700+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:45.339700+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6957,"log_dups_size":0,"ondisk_log_size":6957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28447686,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3614,"num_write_kb":14456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460505822,"num_objects":112437,"num_object_clones":0,"num_object_copies":224874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":112437,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":71572,"num_write_kb":286288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2222,"num_bytes_recovered":9093603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":926474240,"data_stored":926322494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112509,"ondisk_log_size":112509,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533232,"kb_used_data":357152,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93838608,"statfs":{"total":96636764160,"available":96090734592,"internally_reserved":0,"allocated":365723648,"data_stored":365466744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789970,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315032,"kb_used_data":144516,"kb_used_omap":4,"kb_used_meta":170491,"kb_avail":94056808,"statfs":{"total":96636764160,"available":96314171392,"internally_reserved":0,"allocated":147984384,"data_stored":147680047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":174583471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669400,"kb_used_data":403144,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93702440,"statfs":{"total":96636764160,"available":95951298560,"internally_reserved":0,"allocated":412819456,"data_stored":412558475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413138944,"data_stored":413102814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147685376,"data_stored":147605890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365649920,"data_stored":365613790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:48.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:48.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 132 20 114058 114038 3.37396 1.9375 0.0292998 0.0231485 2024-09-11T01:16:48.831 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:48.832 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:48.861 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-11T01:16:48.446592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460505841,"num_objects":112439,"num_object_clones":0,"num_object_copies":224878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":112439,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":71574,"num_write_kb":286290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2223,"num_bytes_recovered":9093622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112512,"ondisk_log_size":112512,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513324,"kb_used_data":912956,"kb_used_omap":12,"kb_used_meta":600307,"kb_avail":281602196,"statfs":{"total":289910292480,"available":288360648704,"internally_reserved":0,"allocated":934866944,"data_stored":934044722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":614714375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,7,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4981,"num_write_kb":19924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1237475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006387"},"pg_stats":[{"pgid":"2.f","version":"39'6936","reported_seq":7000,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.331548+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:45.331548+0000","last_peered":"2024-09-11T01:16:45.331548+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:45.331548+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6936,"num_write_kb":27744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2813},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7038","reported_seq":7109,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.893260+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:44.893260+0000","last_peered":"2024-09-11T01:16:44.893260+0000","last_clean":"2024-09-11T01:16:44.893260+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:44.893260+0000","last_undegraded":"2024-09-11T01:16:44.893260+0000","last_fullsized":"2024-09-11T01:16:44.893260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7038,"log_dups_size":0,"ondisk_log_size":7038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28804038,"num_objects":7034,"num_object_clones":0,"num_object_copies":14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5136,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6936","reported_seq":7009,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.890655+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:44.890655+0000","last_peered":"2024-09-11T01:16:44.890655+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:44.890655+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28393472,"num_objects":6932,"num_object_clones":0,"num_object_copies":13864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6985","reported_seq":7021,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.913374+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:44.913374+0000","last_peered":"2024-09-11T01:16:44.913374+0000","last_clean":"2024-09-11T01:16:44.913374+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:44.913374+0000","last_undegraded":"2024-09-11T01:16:44.913374+0000","last_fullsized":"2024-09-11T01:16:44.913374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6985,"log_dups_size":0,"ondisk_log_size":6985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7007","reported_seq":7119,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.295461+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:45.295461+0000","last_peered":"2024-09-11T01:16:45.295461+0000","last_clean":"2024-09-11T01:16:45.295461+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:45.295461+0000","last_undegraded":"2024-09-11T01:16:45.295461+0000","last_fullsized":"2024-09-11T01:16:45.295461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7007,"log_dups_size":0,"ondisk_log_size":7007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672483,"num_objects":7001,"num_object_clones":0,"num_object_copies":14002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3805,"num_write_kb":15220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7026","reported_seq":7151,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.245095+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:45.245095+0000","last_peered":"2024-09-11T01:16:45.245095+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:45.245095+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7026,"log_dups_size":0,"ondisk_log_size":7026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758499,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3514,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2907},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7195","reported_seq":7319,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.316711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:45.316711+0000","last_peered":"2024-09-11T01:16:45.316711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:45.316711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7195,"log_dups_size":0,"ondisk_log_size":7195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29447110,"num_objects":7191,"num_object_clones":0,"num_object_copies":14382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7191,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7052","reported_seq":7162,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.362986+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:45.362986+0000","last_peered":"2024-09-11T01:16:45.362986+0000","last_clean":"2024-09-11T01:16:45.362986+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:45.362986+0000","last_undegraded":"2024-09-11T01:16:45.362986+0000","last_fullsized":"2024-09-11T01:16:45.362986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28876800,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4005,"num_write_kb":16020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6997","reported_seq":11513,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:44.977911+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:44.977911+0000","last_peered":"2024-09-11T01:16:44.977911+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:44.977911+0000","last_undegraded":"2024-09-11T01:16:44.977911+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28631523,"num_objects":6991,"num_object_clones":0,"num_object_copies":13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":6991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4864,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2221,"num_bytes_recovered":9089507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1904},{"shards":"2","objects":5087}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7072","reported_seq":7191,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.311555+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:45.311555+0000","last_peered":"2024-09-11T01:16:45.311555+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:45.311555+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7072,"log_dups_size":0,"ondisk_log_size":7072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28934144,"num_objects":7064,"num_object_clones":0,"num_object_copies":14128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6909","reported_seq":6994,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.958566+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:44.958566+0000","last_peered":"2024-09-11T01:16:44.958566+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:44.958566+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6909,"log_dups_size":0,"ondisk_log_size":6909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28279267,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4678,"num_write_kb":18712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7118","reported_seq":7202,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.943586+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:44.943586+0000","last_peered":"2024-09-11T01:16:44.943586+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:44.943586+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":19020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7030","reported_seq":7150,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.348326+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:45.348326+0000","last_peered":"2024-09-11T01:16:45.348326+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:45.348326+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28786688,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3720,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7155","reported_seq":7280,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.314182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:45.314182+0000","last_peered":"2024-09-11T01:16:45.314182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:45.314182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7155,"log_dups_size":0,"ondisk_log_size":7155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3548,"num_write_kb":14192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2955},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7096","reported_seq":7218,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.351339+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:45.351339+0000","last_peered":"2024-09-11T01:16:45.351339+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:45.351339+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7096,"log_dups_size":0,"ondisk_log_size":7096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29048832,"num_objects":7092,"num_object_clones":0,"num_object_copies":14184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2944},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6957","reported_seq":7080,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.339700+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:45.339700+0000","last_peered":"2024-09-11T01:16:45.339700+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:45.339700+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6957,"log_dups_size":0,"ondisk_log_size":6957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28447686,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3614,"num_write_kb":14456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460505822,"num_objects":112437,"num_object_clones":0,"num_object_copies":224874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":112437,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":71572,"num_write_kb":286288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2222,"num_bytes_recovered":9093603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":934936576,"data_stored":934784830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112509,"ondisk_log_size":112509,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533232,"kb_used_data":357152,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93838608,"statfs":{"total":96636764160,"available":96090734592,"internally_reserved":0,"allocated":365723648,"data_stored":365466744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789971,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310692,"kb_used_data":152660,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94061148,"statfs":{"total":96636764160,"available":96318615552,"internally_reserved":0,"allocated":156323840,"data_stored":156019503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161803951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669400,"kb_used_data":403144,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93702440,"statfs":{"total":96636764160,"available":95951298560,"internally_reserved":0,"allocated":412819456,"data_stored":412558475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413138944,"data_stored":413102814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156147712,"data_stored":156068226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365649920,"data_stored":365613790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:48.863 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:48.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'6936", 'reported_seq': 7000, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.331548+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:45.331548+0000', 'last_peered': '2024-09-11T01:16:45.331548+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:45.331548+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 6936, 'log_dups_size': 0, 'ondisk_log_size': 6936, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28409856, 'num_objects': 6936, 'num_object_clones': 0, 'num_object_copies': 13872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6936, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6936, 'num_write_kb': 27744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2813}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:48.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'6936", 'reported_seq': 7009, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:44.890655+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:44.890655+0000', 'last_peered': '2024-09-11T01:16:44.890655+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:44.890655+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 6936, 'log_dups_size': 0, 'ondisk_log_size': 6936, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28393472, 'num_objects': 6932, 'num_object_clones': 0, 'num_object_copies': 13864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6932, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4997, 'num_write_kb': 19988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 2927}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:48.864 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:48.864 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7026", 'reported_seq': 7151, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.245095+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:45.245095+0000', 'last_peered': '2024-09-11T01:16:45.245095+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:45.245095+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7026, 'log_dups_size': 0, 'ondisk_log_size': 7026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28758499, 'num_objects': 7022, 'num_object_clones': 0, 'num_object_copies': 14044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7022, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3514, 'num_write_kb': 14056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2907}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:48.865 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'7195", 'reported_seq': 7319, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.316711+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:45.316711+0000', 'last_peered': '2024-09-11T01:16:45.316711+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:45.316711+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 7195, 'log_dups_size': 0, 'ondisk_log_size': 7195, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29447110, 'num_objects': 7191, 'num_object_clones': 0, 'num_object_copies': 14382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7191, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 3948, 'num_write_kb': 15792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2908}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.865 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:48.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'6997", 'reported_seq': 11513, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:44.977911+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:44.977911+0000', 'last_peered': '2024-09-11T01:16:44.977911+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:44.977911+0000', 'last_undegraded': '2024-09-11T01:16:44.977911+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 6997, 'log_dups_size': 0, 'ondisk_log_size': 6997, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28631523, 'num_objects': 6991, 'num_object_clones': 0, 'num_object_copies': 13982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1904, 'num_objects_unfound': 0, 'num_objects_dirty': 6991, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4864, 'num_write_kb': 19456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2221, 'num_bytes_recovered': 9089507, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1904}, {'shards': '2', 'objects': 5087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:48.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:48.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'7072", 'reported_seq': 7191, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.311555+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:45.311555+0000', 'last_peered': '2024-09-11T01:16:45.311555+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:45.311555+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 7072, 'log_dups_size': 0, 'ondisk_log_size': 7072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28934144, 'num_objects': 7064, 'num_object_clones': 0, 'num_object_copies': 14128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3436, 'num_write_kb': 13744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2959}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:48.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'6909", 'reported_seq': 6994, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:44.958566+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:44.958566+0000', 'last_peered': '2024-09-11T01:16:44.958566+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:44.958566+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 6909, 'log_dups_size': 0, 'ondisk_log_size': 6909, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28279267, 'num_objects': 6905, 'num_object_clones': 0, 'num_object_copies': 13810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6905, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4678, 'num_write_kb': 18712, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 2878}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:48.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:48.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'7118", 'reported_seq': 7202, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:44.943586+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:44.943586+0000', 'last_peered': '2024-09-11T01:16:44.943586+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:44.943586+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 7118, 'log_dups_size': 0, 'ondisk_log_size': 7118, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29130752, 'num_objects': 7112, 'num_object_clones': 0, 'num_object_copies': 14224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7112, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4755, 'num_write_kb': 19020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 2935}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:48.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:48.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7030", 'reported_seq': 7150, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.348326+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:45.348326+0000', 'last_peered': '2024-09-11T01:16:45.348326+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:45.348326+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7030, 'log_dups_size': 0, 'ondisk_log_size': 7030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28786688, 'num_objects': 7028, 'num_object_clones': 0, 'num_object_copies': 14056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7028, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3720, 'num_write_kb': 14880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2864}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:48.870 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'7155", 'reported_seq': 7280, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.314182+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:45.314182+0000', 'last_peered': '2024-09-11T01:16:45.314182+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:45.314182+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 7155, 'log_dups_size': 0, 'ondisk_log_size': 7155, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29274112, 'num_objects': 7147, 'num_object_clones': 0, 'num_object_copies': 14294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3548, 'num_write_kb': 14192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2955}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:48.870 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7096", 'reported_seq': 7218, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.351339+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:45.351339+0000', 'last_peered': '2024-09-11T01:16:45.351339+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:45.351339+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7096, 'log_dups_size': 0, 'ondisk_log_size': 7096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29048832, 'num_objects': 7092, 'num_object_clones': 0, 'num_object_copies': 14184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3631, 'num_write_kb': 14524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2944}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:48.870 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'6957", 'reported_seq': 7080, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.339700+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:45.339700+0000', 'last_peered': '2024-09-11T01:16:45.339700+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:45.339700+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 6957, 'log_dups_size': 0, 'ondisk_log_size': 6957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28447686, 'num_objects': 6947, 'num_object_clones': 0, 'num_object_copies': 13894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6947, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3614, 'num_write_kb': 14456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2793}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:48.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:49.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 133 20 114513 114493 3.36196 1.77734 0.0296678 0.0232305 2024-09-11T01:16:49.572 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:49.572 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/224878 objects degraded (20.237%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":210,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112439,"data_bytes":460505841,"bytes_used":1549643776,"bytes_avail":288360648704,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":224878,"degraded_ratio":0.20236750593655228,"misplaced_objects":1904,"misplaced_total":224878,"misplaced_ratio":0.0084668131164453622,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103068,"recovering_keys_per_sec":0,"num_objects_recovered":304,"num_bytes_recovered":1237475,"num_keys_recovered":0,"write_bytes_sec":1699276,"read_op_per_sec":0,"write_op_per_sec":414},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:49.603 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:49.603 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:50.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:50.165 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:50.193 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":137,"stamp":"2024-09-11T01:16:48.446592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460505841,"num_objects":112439,"num_object_clones":0,"num_object_copies":224878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":112439,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":71574,"num_write_kb":286290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2223,"num_bytes_recovered":9093622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112512,"ondisk_log_size":112512,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513324,"kb_used_data":912956,"kb_used_omap":12,"kb_used_meta":600307,"kb_avail":281602196,"statfs":{"total":289910292480,"available":288360648704,"internally_reserved":0,"allocated":934866944,"data_stored":934044722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":614714375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,7,8,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20402176,"num_objects":4981,"num_object_clones":0,"num_object_copies":9962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-303,"num_objects_unfound":0,"num_objects_dirty":4981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4981,"num_write_kb":19924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":1237475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006387"},"pg_stats":[{"pgid":"2.f","version":"39'6936","reported_seq":7000,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.331548+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:45.331548+0000","last_peered":"2024-09-11T01:16:45.331548+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:45.331548+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6936,"num_write_kb":27744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2813},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7038","reported_seq":7109,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.893260+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:44.893260+0000","last_peered":"2024-09-11T01:16:44.893260+0000","last_clean":"2024-09-11T01:16:44.893260+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:44.893260+0000","last_undegraded":"2024-09-11T01:16:44.893260+0000","last_fullsized":"2024-09-11T01:16:44.893260+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7038,"log_dups_size":0,"ondisk_log_size":7038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28804038,"num_objects":7034,"num_object_clones":0,"num_object_copies":14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7034,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5136,"num_write_kb":20544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'6936","reported_seq":7009,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.890655+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:44.890655+0000","last_peered":"2024-09-11T01:16:44.890655+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:44.890655+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":6936,"log_dups_size":0,"ondisk_log_size":6936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28393472,"num_objects":6932,"num_object_clones":0,"num_object_copies":13864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":2927}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'6985","reported_seq":7021,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.913374+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:44.913374+0000","last_peered":"2024-09-11T01:16:44.913374+0000","last_clean":"2024-09-11T01:16:44.913374+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:44.913374+0000","last_undegraded":"2024-09-11T01:16:44.913374+0000","last_fullsized":"2024-09-11T01:16:44.913374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":6985,"log_dups_size":0,"ondisk_log_size":6985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28610560,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7007","reported_seq":7119,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.295461+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:45.295461+0000","last_peered":"2024-09-11T01:16:45.295461+0000","last_clean":"2024-09-11T01:16:45.295461+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:45.295461+0000","last_undegraded":"2024-09-11T01:16:45.295461+0000","last_fullsized":"2024-09-11T01:16:45.295461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7007,"log_dups_size":0,"ondisk_log_size":7007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672483,"num_objects":7001,"num_object_clones":0,"num_object_copies":14002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7001,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3805,"num_write_kb":15220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7026","reported_seq":7151,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.245095+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:45.245095+0000","last_peered":"2024-09-11T01:16:45.245095+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:45.245095+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7026,"log_dups_size":0,"ondisk_log_size":7026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28758499,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3514,"num_write_kb":14056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2907},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7195","reported_seq":7319,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.316711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:45.316711+0000","last_peered":"2024-09-11T01:16:45.316711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:45.316711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7195,"log_dups_size":0,"ondisk_log_size":7195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29447110,"num_objects":7191,"num_object_clones":0,"num_object_copies":14382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7191,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7052","reported_seq":7162,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.362986+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:45.362986+0000","last_peered":"2024-09-11T01:16:45.362986+0000","last_clean":"2024-09-11T01:16:45.362986+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:45.362986+0000","last_undegraded":"2024-09-11T01:16:45.362986+0000","last_fullsized":"2024-09-11T01:16:45.362986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28876800,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4005,"num_write_kb":16020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'6997","reported_seq":11513,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:44.977911+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:44.977911+0000","last_peered":"2024-09-11T01:16:44.977911+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:44.977911+0000","last_undegraded":"2024-09-11T01:16:44.977911+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28631523,"num_objects":6991,"num_object_clones":0,"num_object_copies":13982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":6991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4864,"num_write_kb":19456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2221,"num_bytes_recovered":9089507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1904},{"shards":"2","objects":5087}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7072","reported_seq":7191,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.311555+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:45.311555+0000","last_peered":"2024-09-11T01:16:45.311555+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:45.311555+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7072,"log_dups_size":0,"ondisk_log_size":7072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28934144,"num_objects":7064,"num_object_clones":0,"num_object_copies":14128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'6909","reported_seq":6994,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.958566+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:44.958566+0000","last_peered":"2024-09-11T01:16:44.958566+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:44.958566+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":6909,"log_dups_size":0,"ondisk_log_size":6909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28279267,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4678,"num_write_kb":18712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":2878}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7118","reported_seq":7202,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.943586+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:44.943586+0000","last_peered":"2024-09-11T01:16:44.943586+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:44.943586+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29130752,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4755,"num_write_kb":19020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":2935}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7030","reported_seq":7150,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.348326+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:45.348326+0000","last_peered":"2024-09-11T01:16:45.348326+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:45.348326+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7030,"log_dups_size":0,"ondisk_log_size":7030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28786688,"num_objects":7028,"num_object_clones":0,"num_object_copies":14056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3720,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7155","reported_seq":7280,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.314182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:45.314182+0000","last_peered":"2024-09-11T01:16:45.314182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:45.314182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7155,"log_dups_size":0,"ondisk_log_size":7155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3548,"num_write_kb":14192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2955},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7096","reported_seq":7218,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.351339+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:45.351339+0000","last_peered":"2024-09-11T01:16:45.351339+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:45.351339+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7096,"log_dups_size":0,"ondisk_log_size":7096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29048832,"num_objects":7092,"num_object_clones":0,"num_object_copies":14184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3631,"num_write_kb":14524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2944},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'6957","reported_seq":7080,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.339700+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:45.339700+0000","last_peered":"2024-09-11T01:16:45.339700+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:45.339700+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":6957,"log_dups_size":0,"ondisk_log_size":6957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28447686,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3614,"num_write_kb":14456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2793},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460505822,"num_objects":112437,"num_object_clones":0,"num_object_copies":224874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1904,"num_objects_unfound":0,"num_objects_dirty":112437,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":71572,"num_write_kb":286288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2222,"num_bytes_recovered":9093603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":934936576,"data_stored":934784830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112509,"ondisk_log_size":112509,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533232,"kb_used_data":357152,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93838608,"statfs":{"total":96636764160,"available":96090734592,"internally_reserved":0,"allocated":365723648,"data_stored":365466744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":180285102},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789971,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310692,"kb_used_data":152660,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94061148,"statfs":{"total":96636764160,"available":96318615552,"internally_reserved":0,"allocated":156323840,"data_stored":156019503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161803951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669400,"kb_used_data":403144,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93702440,"statfs":{"total":96636764160,"available":95951298560,"internally_reserved":0,"allocated":412819456,"data_stored":412558475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":413138944,"data_stored":413102814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156147712,"data_stored":156068226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365649920,"data_stored":365613790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:50.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 134 20 114975 114955 3.35034 1.80469 0.0287384 0.023311 2024-09-11T01:16:51.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 135 20 115506 115486 3.34088 2.07422 0.0244018 0.0233766 2024-09-11T01:16:52.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 136 20 116034 116014 3.33148 2.0625 0.0342338 0.0234436 2024-09-11T01:16:53.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:53.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 137 20 116550 116530 3.32188 2.01562 0.0247661 0.0235105 2024-09-11T01:16:53.780 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:53.783 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:53.806 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-11T01:16:52.450501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470737649,"num_objects":114937,"num_object_clones":0,"num_object_copies":229874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":114937,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":74072,"num_write_kb":296282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2374,"num_bytes_recovered":9712118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115010,"ondisk_log_size":115010,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1570132,"kb_used_data":925348,"kb_used_omap":12,"kb_used_meta":644723,"kb_avail":281545388,"statfs":{"total":289910292480,"available":288302477312,"internally_reserved":0,"allocated":947556352,"data_stored":946734130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":660196359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,12,2,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5025,"num_write_kb":20100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009355"},"pg_stats":[{"pgid":"2.f","version":"39'7076","reported_seq":7140,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.314905+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:50.314905+0000","last_peered":"2024-09-11T01:16:50.314905+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:50.314905+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7076,"log_dups_size":0,"ondisk_log_size":7076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983296,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7076,"num_write_kb":28304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2953},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7199","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.964223+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:49.964223+0000","last_peered":"2024-09-11T01:16:49.964223+0000","last_clean":"2024-09-11T01:16:49.964223+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:49.964223+0000","last_undegraded":"2024-09-11T01:16:49.964223+0000","last_fullsized":"2024-09-11T01:16:49.964223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7199,"log_dups_size":0,"ondisk_log_size":7199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29463494,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7080","reported_seq":7153,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.873324+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:49.873324+0000","last_peered":"2024-09-11T01:16:49.873324+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:49.873324+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7080,"log_dups_size":0,"ondisk_log_size":7080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983296,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":20564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7123","reported_seq":7159,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.924699+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:49.924699+0000","last_peered":"2024-09-11T01:16:49.924699+0000","last_clean":"2024-09-11T01:16:49.924699+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:49.924699+0000","last_undegraded":"2024-09-11T01:16:49.924699+0000","last_fullsized":"2024-09-11T01:16:49.924699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29175808,"num_objects":7123,"num_object_clones":0,"num_object_copies":14246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7123,"num_write_kb":28492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7150","reported_seq":7262,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.401313+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:50.401313+0000","last_peered":"2024-09-11T01:16:50.401313+0000","last_clean":"2024-09-11T01:16:50.401313+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:50.401313+0000","last_undegraded":"2024-09-11T01:16:50.401313+0000","last_fullsized":"2024-09-11T01:16:50.401313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29258211,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7168","reported_seq":7293,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.348579+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:50.348579+0000","last_peered":"2024-09-11T01:16:50.348579+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:50.348579+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29340131,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3656,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7358","reported_seq":7482,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.360134+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:50.360134+0000","last_peered":"2024-09-11T01:16:50.360134+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:50.360134+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30114758,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4111,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3071},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7214","reported_seq":7324,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.313938+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:50.313938+0000","last_peered":"2024-09-11T01:16:50.313938+0000","last_clean":"2024-09-11T01:16:50.313938+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:50.313938+0000","last_undegraded":"2024-09-11T01:16:50.313938+0000","last_fullsized":"2024-09-11T01:16:50.313938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7214,"log_dups_size":0,"ondisk_log_size":7214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29540352,"num_objects":7212,"num_object_clones":0,"num_object_copies":14424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7172","reported_seq":11990,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:49.965645+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:49.965645+0000","last_peered":"2024-09-11T01:16:49.965645+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:49.965645+0000","last_undegraded":"2024-09-11T01:16:49.965645+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29348323,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5039,"num_write_kb":20156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2372,"num_bytes_recovered":9708003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1753},{"shards":"2","objects":5413}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7252","reported_seq":7368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.391585+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:50.391585+0000","last_peered":"2024-09-11T01:16:50.391585+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:50.391585+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671424,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3139},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7054","reported_seq":7139,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.908891+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:49.908891+0000","last_peered":"2024-09-11T01:16:49.908891+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:49.908891+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7054,"log_dups_size":0,"ondisk_log_size":7054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4823,"num_write_kb":19292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7295","reported_seq":7379,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.899363+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:49.899363+0000","last_peered":"2024-09-11T01:16:49.899363+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:49.899363+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7295,"log_dups_size":0,"ondisk_log_size":7295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4932,"num_write_kb":19728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3112}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7178","reported_seq":7298,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.356911+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:50.356911+0000","last_peered":"2024-09-11T01:16:50.356911+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:50.356911+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7178,"log_dups_size":0,"ondisk_log_size":7178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3868,"num_write_kb":15472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3012},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7321","reported_seq":7446,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.386756+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:50.386756+0000","last_peered":"2024-09-11T01:16:50.386756+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:50.386756+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7321,"log_dups_size":0,"ondisk_log_size":7321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3714,"num_write_kb":14856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7257","reported_seq":7381,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.384156+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:50.384156+0000","last_peered":"2024-09-11T01:16:50.384156+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:50.384156+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29708288,"num_objects":7253,"num_object_clones":0,"num_object_copies":14506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3105},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7110","reported_seq":7235,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.317805+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:50.317805+0000","last_peered":"2024-09-11T01:16:50.317805+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:50.317805+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7110,"log_dups_size":0,"ondisk_log_size":7110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29074374,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3767,"num_write_kb":15068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2946},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470737630,"num_objects":114935,"num_object_clones":0,"num_object_copies":229870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":114935,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":74070,"num_write_kb":296280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2373,"num_bytes_recovered":9712099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":947654656,"data_stored":947502910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115007,"ondisk_log_size":115007,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578520,"kb_used_data":362184,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93793320,"statfs":{"total":96636764160,"available":96044359680,"internally_reserved":0,"allocated":370876416,"data_stored":370619512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789971,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310692,"kb_used_data":152660,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94061148,"statfs":{"total":96636764160,"available":96318615552,"internally_reserved":0,"allocated":156323840,"data_stored":156019503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161803951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680920,"kb_used_data":410504,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93690920,"statfs":{"total":96636764160,"available":95939502080,"internally_reserved":0,"allocated":420356096,"data_stored":420095115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,0,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420753408,"data_stored":420717278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156147712,"data_stored":156068226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370753536,"data_stored":370717406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:53.807 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:54.341 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 138 20 117008 116988 3.31077 1.78906 0.0626899 0.0235899 2024-09-11T01:16:54.392 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:54.393 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:54.419 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-11T01:16:52.450501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470737649,"num_objects":114937,"num_object_clones":0,"num_object_copies":229874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":114937,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":74072,"num_write_kb":296282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2374,"num_bytes_recovered":9712118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115010,"ondisk_log_size":115010,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1570132,"kb_used_data":925348,"kb_used_omap":12,"kb_used_meta":644723,"kb_avail":281545388,"statfs":{"total":289910292480,"available":288302477312,"internally_reserved":0,"allocated":947556352,"data_stored":946734130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":660196359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,12,2,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5025,"num_write_kb":20100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009355"},"pg_stats":[{"pgid":"2.f","version":"39'7076","reported_seq":7140,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.314905+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:50.314905+0000","last_peered":"2024-09-11T01:16:50.314905+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:50.314905+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7076,"log_dups_size":0,"ondisk_log_size":7076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983296,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7076,"num_write_kb":28304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2953},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7199","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.964223+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:49.964223+0000","last_peered":"2024-09-11T01:16:49.964223+0000","last_clean":"2024-09-11T01:16:49.964223+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:49.964223+0000","last_undegraded":"2024-09-11T01:16:49.964223+0000","last_fullsized":"2024-09-11T01:16:49.964223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7199,"log_dups_size":0,"ondisk_log_size":7199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29463494,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7080","reported_seq":7153,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.873324+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:49.873324+0000","last_peered":"2024-09-11T01:16:49.873324+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:49.873324+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7080,"log_dups_size":0,"ondisk_log_size":7080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983296,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":20564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7123","reported_seq":7159,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.924699+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:49.924699+0000","last_peered":"2024-09-11T01:16:49.924699+0000","last_clean":"2024-09-11T01:16:49.924699+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:49.924699+0000","last_undegraded":"2024-09-11T01:16:49.924699+0000","last_fullsized":"2024-09-11T01:16:49.924699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29175808,"num_objects":7123,"num_object_clones":0,"num_object_copies":14246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7123,"num_write_kb":28492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7150","reported_seq":7262,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.401313+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:50.401313+0000","last_peered":"2024-09-11T01:16:50.401313+0000","last_clean":"2024-09-11T01:16:50.401313+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:50.401313+0000","last_undegraded":"2024-09-11T01:16:50.401313+0000","last_fullsized":"2024-09-11T01:16:50.401313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29258211,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7168","reported_seq":7293,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.348579+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:50.348579+0000","last_peered":"2024-09-11T01:16:50.348579+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:50.348579+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29340131,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3656,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7358","reported_seq":7482,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.360134+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:50.360134+0000","last_peered":"2024-09-11T01:16:50.360134+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:50.360134+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30114758,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4111,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3071},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7214","reported_seq":7324,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.313938+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:50.313938+0000","last_peered":"2024-09-11T01:16:50.313938+0000","last_clean":"2024-09-11T01:16:50.313938+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:50.313938+0000","last_undegraded":"2024-09-11T01:16:50.313938+0000","last_fullsized":"2024-09-11T01:16:50.313938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7214,"log_dups_size":0,"ondisk_log_size":7214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29540352,"num_objects":7212,"num_object_clones":0,"num_object_copies":14424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7172","reported_seq":11990,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:49.965645+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:49.965645+0000","last_peered":"2024-09-11T01:16:49.965645+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:49.965645+0000","last_undegraded":"2024-09-11T01:16:49.965645+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29348323,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5039,"num_write_kb":20156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2372,"num_bytes_recovered":9708003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1753},{"shards":"2","objects":5413}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7252","reported_seq":7368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.391585+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:50.391585+0000","last_peered":"2024-09-11T01:16:50.391585+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:50.391585+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671424,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3139},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7054","reported_seq":7139,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.908891+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:49.908891+0000","last_peered":"2024-09-11T01:16:49.908891+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:49.908891+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7054,"log_dups_size":0,"ondisk_log_size":7054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4823,"num_write_kb":19292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7295","reported_seq":7379,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.899363+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:49.899363+0000","last_peered":"2024-09-11T01:16:49.899363+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:49.899363+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7295,"log_dups_size":0,"ondisk_log_size":7295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4932,"num_write_kb":19728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3112}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7178","reported_seq":7298,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.356911+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:50.356911+0000","last_peered":"2024-09-11T01:16:50.356911+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:50.356911+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7178,"log_dups_size":0,"ondisk_log_size":7178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3868,"num_write_kb":15472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3012},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7321","reported_seq":7446,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.386756+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:50.386756+0000","last_peered":"2024-09-11T01:16:50.386756+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:50.386756+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7321,"log_dups_size":0,"ondisk_log_size":7321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3714,"num_write_kb":14856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7257","reported_seq":7381,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.384156+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:50.384156+0000","last_peered":"2024-09-11T01:16:50.384156+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:50.384156+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29708288,"num_objects":7253,"num_object_clones":0,"num_object_copies":14506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3105},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7110","reported_seq":7235,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.317805+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:50.317805+0000","last_peered":"2024-09-11T01:16:50.317805+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:50.317805+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7110,"log_dups_size":0,"ondisk_log_size":7110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29074374,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3767,"num_write_kb":15068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2946},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470737630,"num_objects":114935,"num_object_clones":0,"num_object_copies":229870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":114935,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":74070,"num_write_kb":296280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2373,"num_bytes_recovered":9712099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":947654656,"data_stored":947502910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115007,"ondisk_log_size":115007,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578520,"kb_used_data":362184,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93793320,"statfs":{"total":96636764160,"available":96044359680,"internally_reserved":0,"allocated":370876416,"data_stored":370619512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789971,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310692,"kb_used_data":152660,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":94061148,"statfs":{"total":96636764160,"available":96318615552,"internally_reserved":0,"allocated":156323840,"data_stored":156019503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161803951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680920,"kb_used_data":410504,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93690920,"statfs":{"total":96636764160,"available":95939502080,"internally_reserved":0,"allocated":420356096,"data_stored":420095115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,0,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420753408,"data_stored":420717278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156147712,"data_stored":156068226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370753536,"data_stored":370717406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'7076", 'reported_seq': 7140, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.314905+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:50.314905+0000', 'last_peered': '2024-09-11T01:16:50.314905+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:50.314905+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 7076, 'log_dups_size': 0, 'ondisk_log_size': 7076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28983296, 'num_objects': 7076, 'num_object_clones': 0, 'num_object_copies': 14152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7076, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7076, 'num_write_kb': 28304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2953}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'7080", 'reported_seq': 7153, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:49.873324+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:49.873324+0000', 'last_peered': '2024-09-11T01:16:49.873324+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:49.873324+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 7080, 'log_dups_size': 0, 'ondisk_log_size': 7080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28983296, 'num_objects': 7076, 'num_object_clones': 0, 'num_object_copies': 14152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7076, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5141, 'num_write_kb': 20564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 3071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7168", 'reported_seq': 7293, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.348579+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:50.348579+0000', 'last_peered': '2024-09-11T01:16:50.348579+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:50.348579+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7168, 'log_dups_size': 0, 'ondisk_log_size': 7168, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29340131, 'num_objects': 7164, 'num_object_clones': 0, 'num_object_copies': 14328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7164, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3656, 'num_write_kb': 14624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3049}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:54.421 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'7358", 'reported_seq': 7482, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.360134+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:50.360134+0000', 'last_peered': '2024-09-11T01:16:50.360134+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:50.360134+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 7358, 'log_dups_size': 0, 'ondisk_log_size': 7358, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30114758, 'num_objects': 7354, 'num_object_clones': 0, 'num_object_copies': 14708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7354, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4111, 'num_write_kb': 16444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3071}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'7172", 'reported_seq': 11990, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:49.965645+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:49.965645+0000', 'last_peered': '2024-09-11T01:16:49.965645+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:49.965645+0000', 'last_undegraded': '2024-09-11T01:16:49.965645+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 7172, 'log_dups_size': 0, 'ondisk_log_size': 7172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29348323, 'num_objects': 7166, 'num_object_clones': 0, 'num_object_copies': 14332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1753, 'num_objects_unfound': 0, 'num_objects_dirty': 7166, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5039, 'num_write_kb': 20156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2372, 'num_bytes_recovered': 9708003, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1753}, {'shards': '2', 'objects': 5413}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'7252", 'reported_seq': 7368, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.391585+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:50.391585+0000', 'last_peered': '2024-09-11T01:16:50.391585+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:50.391585+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 7252, 'log_dups_size': 0, 'ondisk_log_size': 7252, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29671424, 'num_objects': 7244, 'num_object_clones': 0, 'num_object_copies': 14488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3616, 'num_write_kb': 14464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3139}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'7054", 'reported_seq': 7139, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:49.908891+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:49.908891+0000', 'last_peered': '2024-09-11T01:16:49.908891+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:49.908891+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 7054, 'log_dups_size': 0, 'ondisk_log_size': 7054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28873187, 'num_objects': 7050, 'num_object_clones': 0, 'num_object_copies': 14100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7050, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4823, 'num_write_kb': 19292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 3023}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:54.422 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'7295", 'reported_seq': 7379, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:49.899363+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:49.899363+0000', 'last_peered': '2024-09-11T01:16:49.899363+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:49.899363+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 7295, 'log_dups_size': 0, 'ondisk_log_size': 7295, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29855744, 'num_objects': 7289, 'num_object_clones': 0, 'num_object_copies': 14578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4932, 'num_write_kb': 19728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 3112}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7178", 'reported_seq': 7298, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.356911+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:50.356911+0000', 'last_peered': '2024-09-11T01:16:50.356911+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:50.356911+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7178, 'log_dups_size': 0, 'ondisk_log_size': 7178, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29392896, 'num_objects': 7176, 'num_object_clones': 0, 'num_object_copies': 14352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3868, 'num_write_kb': 15472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3012}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'7321", 'reported_seq': 7446, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.386756+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:50.386756+0000', 'last_peered': '2024-09-11T01:16:50.386756+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:50.386756+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 7321, 'log_dups_size': 0, 'ondisk_log_size': 7321, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29954048, 'num_objects': 7313, 'num_object_clones': 0, 'num_object_copies': 14626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3714, 'num_write_kb': 14856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3121}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7257", 'reported_seq': 7381, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.384156+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:50.384156+0000', 'last_peered': '2024-09-11T01:16:50.384156+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:50.384156+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7257, 'log_dups_size': 0, 'ondisk_log_size': 7257, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29708288, 'num_objects': 7253, 'num_object_clones': 0, 'num_object_copies': 14506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3792, 'num_write_kb': 15168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3105}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:54.423 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'7110", 'reported_seq': 7235, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:50.317805+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:50.317805+0000', 'last_peered': '2024-09-11T01:16:50.317805+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:50.317805+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 7110, 'log_dups_size': 0, 'ondisk_log_size': 7110, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29074374, 'num_objects': 7100, 'num_object_clones': 0, 'num_object_copies': 14200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7100, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3767, 'num_write_kb': 15068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2946}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:54.424 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:55.133 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:55.134 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/229874 objects degraded (19.797%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":215,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114937,"data_bytes":470737649,"bytes_used":1638821888,"bytes_avail":288271470592,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":229874,"degraded_ratio":0.19796932232440378,"misplaced_objects":1753,"misplaced_total":229874,"misplaced_ratio":0.0076259168065983974,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":102677,"recovering_keys_per_sec":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"write_bytes_sec":1714131,"read_op_per_sec":0,"write_op_per_sec":418},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:55.160 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:55.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:55.344 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 139 19 117457 117438 3.29953 1.75781 0.0973373 0.0236688 2024-09-11T01:16:55.727 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:55.727 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:55.748 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-11T01:16:54.450956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":470737649,"num_objects":114937,"num_object_clones":0,"num_object_copies":229874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":114937,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":74072,"num_write_kb":296282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2374,"num_bytes_recovered":9712118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115010,"ondisk_log_size":115010,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1600412,"kb_used_data":933420,"kb_used_omap":12,"kb_used_meta":666931,"kb_avail":281515108,"statfs":{"total":289910292480,"available":288271470592,"internally_reserved":0,"allocated":955822080,"data_stored":954999858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":682937351},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,11,3,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20582400,"num_objects":5025,"num_object_clones":0,"num_object_copies":10050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":5025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5025,"num_write_kb":20100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007477"},"pg_stats":[{"pgid":"2.f","version":"39'7076","reported_seq":7140,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.314905+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:50.314905+0000","last_peered":"2024-09-11T01:16:50.314905+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:50.314905+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7076,"log_dups_size":0,"ondisk_log_size":7076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983296,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7076,"num_write_kb":28304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2953},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7199","reported_seq":7270,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.964223+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:49.964223+0000","last_peered":"2024-09-11T01:16:49.964223+0000","last_clean":"2024-09-11T01:16:49.964223+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:49.964223+0000","last_undegraded":"2024-09-11T01:16:49.964223+0000","last_fullsized":"2024-09-11T01:16:49.964223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7199,"log_dups_size":0,"ondisk_log_size":7199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29463494,"num_objects":7195,"num_object_clones":0,"num_object_copies":14390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7195,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5297,"num_write_kb":21188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7080","reported_seq":7153,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.873324+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:49.873324+0000","last_peered":"2024-09-11T01:16:49.873324+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:49.873324+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7080,"log_dups_size":0,"ondisk_log_size":7080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28983296,"num_objects":7076,"num_object_clones":0,"num_object_copies":14152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5141,"num_write_kb":20564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7123","reported_seq":7159,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.924699+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:49.924699+0000","last_peered":"2024-09-11T01:16:49.924699+0000","last_clean":"2024-09-11T01:16:49.924699+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:49.924699+0000","last_undegraded":"2024-09-11T01:16:49.924699+0000","last_fullsized":"2024-09-11T01:16:49.924699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7123,"log_dups_size":0,"ondisk_log_size":7123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29175808,"num_objects":7123,"num_object_clones":0,"num_object_copies":14246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7123,"num_write_kb":28492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7150","reported_seq":7262,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.401313+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:50.401313+0000","last_peered":"2024-09-11T01:16:50.401313+0000","last_clean":"2024-09-11T01:16:50.401313+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:50.401313+0000","last_undegraded":"2024-09-11T01:16:50.401313+0000","last_fullsized":"2024-09-11T01:16:50.401313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7150,"log_dups_size":0,"ondisk_log_size":7150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29258211,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7168","reported_seq":7293,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.348579+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:50.348579+0000","last_peered":"2024-09-11T01:16:50.348579+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:50.348579+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7168,"log_dups_size":0,"ondisk_log_size":7168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29340131,"num_objects":7164,"num_object_clones":0,"num_object_copies":14328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7164,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3656,"num_write_kb":14624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7358","reported_seq":7482,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.360134+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:50.360134+0000","last_peered":"2024-09-11T01:16:50.360134+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:50.360134+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30114758,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4111,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3071},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7214","reported_seq":7324,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.313938+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:50.313938+0000","last_peered":"2024-09-11T01:16:50.313938+0000","last_clean":"2024-09-11T01:16:50.313938+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:50.313938+0000","last_undegraded":"2024-09-11T01:16:50.313938+0000","last_fullsized":"2024-09-11T01:16:50.313938+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7214,"log_dups_size":0,"ondisk_log_size":7214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29540352,"num_objects":7212,"num_object_clones":0,"num_object_copies":14424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7172","reported_seq":11990,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:49.965645+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:49.965645+0000","last_peered":"2024-09-11T01:16:49.965645+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:49.965645+0000","last_undegraded":"2024-09-11T01:16:49.965645+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7172,"log_dups_size":0,"ondisk_log_size":7172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29348323,"num_objects":7166,"num_object_clones":0,"num_object_copies":14332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":7166,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5039,"num_write_kb":20156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2372,"num_bytes_recovered":9708003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1753},{"shards":"2","objects":5413}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7252","reported_seq":7368,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.391585+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:50.391585+0000","last_peered":"2024-09-11T01:16:50.391585+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:50.391585+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7252,"log_dups_size":0,"ondisk_log_size":7252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671424,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3139},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7054","reported_seq":7139,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.908891+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:49.908891+0000","last_peered":"2024-09-11T01:16:49.908891+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:49.908891+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7054,"log_dups_size":0,"ondisk_log_size":7054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4823,"num_write_kb":19292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3023}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7295","reported_seq":7379,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.899363+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:49.899363+0000","last_peered":"2024-09-11T01:16:49.899363+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:49.899363+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7295,"log_dups_size":0,"ondisk_log_size":7295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4932,"num_write_kb":19728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3112}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7178","reported_seq":7298,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.356911+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:50.356911+0000","last_peered":"2024-09-11T01:16:50.356911+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:50.356911+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7178,"log_dups_size":0,"ondisk_log_size":7178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29392896,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3868,"num_write_kb":15472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3012},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7321","reported_seq":7446,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.386756+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:50.386756+0000","last_peered":"2024-09-11T01:16:50.386756+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:50.386756+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7321,"log_dups_size":0,"ondisk_log_size":7321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954048,"num_objects":7313,"num_object_clones":0,"num_object_copies":14626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3714,"num_write_kb":14856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3121},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7257","reported_seq":7381,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.384156+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:50.384156+0000","last_peered":"2024-09-11T01:16:50.384156+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:50.384156+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7257,"log_dups_size":0,"ondisk_log_size":7257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29708288,"num_objects":7253,"num_object_clones":0,"num_object_copies":14506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3792,"num_write_kb":15168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3105},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7110","reported_seq":7235,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.317805+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:50.317805+0000","last_peered":"2024-09-11T01:16:50.317805+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:50.317805+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7110,"log_dups_size":0,"ondisk_log_size":7110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29074374,"num_objects":7100,"num_object_clones":0,"num_object_copies":14200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3767,"num_write_kb":15068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2946},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":470737630,"num_objects":114935,"num_object_clones":0,"num_object_copies":229870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1753,"num_objects_unfound":0,"num_objects_dirty":114935,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":74070,"num_write_kb":296280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2373,"num_bytes_recovered":9712099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":955961344,"data_stored":955809598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115007,"ondisk_log_size":115007,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578520,"kb_used_data":362184,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93793320,"statfs":{"total":96636764160,"available":96044359680,"internally_reserved":0,"allocated":370876416,"data_stored":370619512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":221507246},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789972,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340972,"kb_used_data":160732,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94030868,"statfs":{"total":96636764160,"available":96287608832,"internally_reserved":0,"allocated":164589568,"data_stored":164285231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":184544943},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680920,"kb_used_data":410504,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93690920,"statfs":{"total":96636764160,"available":95939502080,"internally_reserved":0,"allocated":420356096,"data_stored":420095115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,0,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":420753408,"data_stored":420717278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":164454400,"data_stored":164374914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370753536,"data_stored":370717406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:56.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:16:56.344452+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0237407 2024-09-11T01:16:56.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:16:56.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 140 20 117954 117934 3.28978 1.9375 0.0404949 0.0237407 2024-09-11T01:16:57.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 141 20 118457 118437 3.28039 1.96484 0.0148354 0.0238076 2024-09-11T01:16:58.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 142 20 118976 118956 3.27156 2.02734 0.0234774 0.0238704 2024-09-11T01:16:58.750 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:59.315 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:59.320 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:59.344 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-11T01:16:58.452125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480891633,"num_objects":117416,"num_object_clones":0,"num_object_copies":234832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1599,"num_objects_unfound":0,"num_objects_dirty":117416,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":76551,"num_write_kb":306198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2528,"num_bytes_recovered":10342902,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117489,"ondisk_log_size":117489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1638900,"kb_used_data":953860,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281476620,"statfs":{"total":289910292480,"available":288232058880,"internally_reserved":0,"allocated":976752640,"data_stored":975930418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":701418503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,7,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006035"},"pg_stats":[{"pgid":"2.f","version":"39'7249","reported_seq":7313,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.411988+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:55.411988+0000","last_peered":"2024-09-11T01:16:55.411988+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:55.411988+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7249,"log_dups_size":0,"ondisk_log_size":7249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7356","reported_seq":7427,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.946737+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:54.946737+0000","last_peered":"2024-09-11T01:16:54.946737+0000","last_clean":"2024-09-11T01:16:54.946737+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:54.946737+0000","last_undegraded":"2024-09-11T01:16:54.946737+0000","last_fullsized":"2024-09-11T01:16:54.946737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30106566,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5454,"num_write_kb":21816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7237","reported_seq":7310,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.985675+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:54.985675+0000","last_peered":"2024-09-11T01:16:54.985675+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:54.985675+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5298,"num_write_kb":21192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7291","reported_seq":7327,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.010340+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:55.010340+0000","last_peered":"2024-09-11T01:16:55.010340+0000","last_clean":"2024-09-11T01:16:55.010340+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:55.010340+0000","last_undegraded":"2024-09-11T01:16:55.010340+0000","last_fullsized":"2024-09-11T01:16:55.010340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7291,"log_dups_size":0,"ondisk_log_size":7291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7291,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7284","reported_seq":7396,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.374985+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:55.374985+0000","last_peered":"2024-09-11T01:16:55.374985+0000","last_clean":"2024-09-11T01:16:55.374985+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:55.374985+0000","last_undegraded":"2024-09-11T01:16:55.374985+0000","last_fullsized":"2024-09-11T01:16:55.374985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29807075,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4082,"num_write_kb":16328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7324","reported_seq":7448,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.361674+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:55.361674+0000","last_peered":"2024-09-11T01:16:55.361674+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:55.361674+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7324,"log_dups_size":0,"ondisk_log_size":7324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3205},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7510","reported_seq":7634,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.397668+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:55.397668+0000","last_peered":"2024-09-11T01:16:55.397668+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:55.397668+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30737350,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4263,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7367","reported_seq":7477,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.334522+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:55.334522+0000","last_peered":"2024-09-11T01:16:55.334522+0000","last_clean":"2024-09-11T01:16:55.334522+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:55.334522+0000","last_undegraded":"2024-09-11T01:16:55.334522+0000","last_fullsized":"2024-09-11T01:16:55.334522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30167040,"num_objects":7365,"num_object_clones":0,"num_object_copies":14730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7320","reported_seq":12446,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:54.975466+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:54.975466+0000","last_peered":"2024-09-11T01:16:54.975466+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:54.975466+0000","last_undegraded":"2024-09-11T01:16:54.975466+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7320,"log_dups_size":0,"ondisk_log_size":7320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954531,"num_objects":7314,"num_object_clones":0,"num_object_copies":14628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1599,"num_objects_unfound":0,"num_objects_dirty":7314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2526,"num_bytes_recovered":10338787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1599},{"shards":"2","objects":5715}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7397","reported_seq":7516,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.357749+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:55.357749+0000","last_peered":"2024-09-11T01:16:55.357749+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:55.357749+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30265344,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3761,"num_write_kb":15044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3284},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7194","reported_seq":7279,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.795264+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:54.795264+0000","last_peered":"2024-09-11T01:16:54.795264+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:54.795264+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446627,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4963,"num_write_kb":19852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7454","reported_seq":7534,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.960025+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:54.960025+0000","last_peered":"2024-09-11T01:16:54.960025+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:54.960025+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7454,"log_dups_size":0,"ondisk_log_size":7454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30507008,"num_objects":7448,"num_object_clones":0,"num_object_copies":14896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5091,"num_write_kb":20364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3271}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7348","reported_seq":7467,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.389955+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:55.389955+0000","last_peered":"2024-09-11T01:16:55.389955+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:55.389955+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7348,"log_dups_size":0,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30089216,"num_objects":7346,"num_object_clones":0,"num_object_copies":14692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7497","reported_seq":7622,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.410027+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:55.410027+0000","last_peered":"2024-09-11T01:16:55.410027+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:55.410027+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7497,"log_dups_size":0,"ondisk_log_size":7497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30674944,"num_objects":7489,"num_object_clones":0,"num_object_copies":14978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3890,"num_write_kb":15560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7394","reported_seq":7517,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.404504+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:55.404504+0000","last_peered":"2024-09-11T01:16:55.404504+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:55.404504+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30269440,"num_objects":7390,"num_object_clones":0,"num_object_copies":14780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7264","reported_seq":7389,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.402808+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:55.402808+0000","last_peered":"2024-09-11T01:16:55.402808+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:55.402808+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29705158,"num_objects":7254,"num_object_clones":0,"num_object_copies":14508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3921,"num_write_kb":15684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480891614,"num_objects":117414,"num_object_clones":0,"num_object_copies":234828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1599,"num_objects_unfound":0,"num_objects_dirty":117414,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":76549,"num_write_kb":306196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2527,"num_bytes_recovered":10342883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":976949248,"data_stored":976797502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117486,"ondisk_log_size":117486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605552,"kb_used_data":367008,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93766288,"statfs":{"total":96636764160,"available":96016678912,"internally_reserved":0,"allocated":375816192,"data_stored":375559288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340600,"kb_used_data":168680,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94031240,"statfs":{"total":96636764160,"available":96287989760,"internally_reserved":0,"allocated":172728320,"data_stored":172423983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692748,"kb_used_data":418172,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93679092,"statfs":{"total":96636764160,"available":95927390208,"internally_reserved":0,"allocated":428208128,"data_stored":427947147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428425216,"data_stored":428389086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172748800,"data_stored":172669314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":375775232,"data_stored":375739102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:59.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:59.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 143 20 119444 119424 3.26147 1.82812 0.0345007 0.0239454 2024-09-11T01:16:59.922 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:16:59.923 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:16:59.955 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-11T01:16:58.452125+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480891633,"num_objects":117416,"num_object_clones":0,"num_object_copies":234832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1599,"num_objects_unfound":0,"num_objects_dirty":117416,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":76551,"num_write_kb":306198,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2528,"num_bytes_recovered":10342902,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117489,"ondisk_log_size":117489,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1638900,"kb_used_data":953860,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281476620,"statfs":{"total":289910292480,"available":288232058880,"internally_reserved":0,"allocated":976752640,"data_stored":975930418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":701418503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,7,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20385792,"num_objects":4977,"num_object_clones":0,"num_object_copies":9954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":4977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4977,"num_write_kb":19908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":1249280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006035"},"pg_stats":[{"pgid":"2.f","version":"39'7249","reported_seq":7313,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.411988+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:55.411988+0000","last_peered":"2024-09-11T01:16:55.411988+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:55.411988+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7249,"log_dups_size":0,"ondisk_log_size":7249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7356","reported_seq":7427,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.946737+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:54.946737+0000","last_peered":"2024-09-11T01:16:54.946737+0000","last_clean":"2024-09-11T01:16:54.946737+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:54.946737+0000","last_undegraded":"2024-09-11T01:16:54.946737+0000","last_fullsized":"2024-09-11T01:16:54.946737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7356,"log_dups_size":0,"ondisk_log_size":7356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30106566,"num_objects":7352,"num_object_clones":0,"num_object_copies":14704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5454,"num_write_kb":21816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7237","reported_seq":7310,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.985675+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:54.985675+0000","last_peered":"2024-09-11T01:16:54.985675+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:54.985675+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7237,"log_dups_size":0,"ondisk_log_size":7237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29626368,"num_objects":7233,"num_object_clones":0,"num_object_copies":14466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5298,"num_write_kb":21192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3228}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7291","reported_seq":7327,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.010340+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:16:55.010340+0000","last_peered":"2024-09-11T01:16:55.010340+0000","last_clean":"2024-09-11T01:16:55.010340+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:16:55.010340+0000","last_undegraded":"2024-09-11T01:16:55.010340+0000","last_fullsized":"2024-09-11T01:16:55.010340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7291,"log_dups_size":0,"ondisk_log_size":7291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7291,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7284","reported_seq":7396,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.374985+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:55.374985+0000","last_peered":"2024-09-11T01:16:55.374985+0000","last_clean":"2024-09-11T01:16:55.374985+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:55.374985+0000","last_undegraded":"2024-09-11T01:16:55.374985+0000","last_fullsized":"2024-09-11T01:16:55.374985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29807075,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4082,"num_write_kb":16328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7324","reported_seq":7448,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.361674+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:55.361674+0000","last_peered":"2024-09-11T01:16:55.361674+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:55.361674+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7324,"log_dups_size":0,"ondisk_log_size":7324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3205},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7510","reported_seq":7634,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.397668+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:55.397668+0000","last_peered":"2024-09-11T01:16:55.397668+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:55.397668+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30737350,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4263,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7367","reported_seq":7477,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.334522+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:55.334522+0000","last_peered":"2024-09-11T01:16:55.334522+0000","last_clean":"2024-09-11T01:16:55.334522+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:55.334522+0000","last_undegraded":"2024-09-11T01:16:55.334522+0000","last_fullsized":"2024-09-11T01:16:55.334522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30167040,"num_objects":7365,"num_object_clones":0,"num_object_copies":14730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7320","reported_seq":12446,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:54.975466+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:16:54.975466+0000","last_peered":"2024-09-11T01:16:54.975466+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:16:54.975466+0000","last_undegraded":"2024-09-11T01:16:54.975466+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7320,"log_dups_size":0,"ondisk_log_size":7320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29954531,"num_objects":7314,"num_object_clones":0,"num_object_copies":14628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1599,"num_objects_unfound":0,"num_objects_dirty":7314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2526,"num_bytes_recovered":10338787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1599},{"shards":"2","objects":5715}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7397","reported_seq":7516,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.357749+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:55.357749+0000","last_peered":"2024-09-11T01:16:55.357749+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:55.357749+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30265344,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3761,"num_write_kb":15044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3284},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7194","reported_seq":7279,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.795264+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:54.795264+0000","last_peered":"2024-09-11T01:16:54.795264+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:54.795264+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29446627,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4963,"num_write_kb":19852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7454","reported_seq":7534,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.960025+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:54.960025+0000","last_peered":"2024-09-11T01:16:54.960025+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:54.960025+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7454,"log_dups_size":0,"ondisk_log_size":7454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30507008,"num_objects":7448,"num_object_clones":0,"num_object_copies":14896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5091,"num_write_kb":20364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3271}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7348","reported_seq":7467,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.389955+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:55.389955+0000","last_peered":"2024-09-11T01:16:55.389955+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:55.389955+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7348,"log_dups_size":0,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30089216,"num_objects":7346,"num_object_clones":0,"num_object_copies":14692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7497","reported_seq":7622,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.410027+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:55.410027+0000","last_peered":"2024-09-11T01:16:55.410027+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:55.410027+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7497,"log_dups_size":0,"ondisk_log_size":7497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30674944,"num_objects":7489,"num_object_clones":0,"num_object_copies":14978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3890,"num_write_kb":15560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7394","reported_seq":7517,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.404504+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:55.404504+0000","last_peered":"2024-09-11T01:16:55.404504+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:55.404504+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30269440,"num_objects":7390,"num_object_clones":0,"num_object_copies":14780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7264","reported_seq":7389,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.402808+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:55.402808+0000","last_peered":"2024-09-11T01:16:55.402808+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:55.402808+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29705158,"num_objects":7254,"num_object_clones":0,"num_object_copies":14508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3921,"num_write_kb":15684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480891614,"num_objects":117414,"num_object_clones":0,"num_object_copies":234828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1599,"num_objects_unfound":0,"num_objects_dirty":117414,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":76549,"num_write_kb":306196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2527,"num_bytes_recovered":10342883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":976949248,"data_stored":976797502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117486,"ondisk_log_size":117486,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605552,"kb_used_data":367008,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93766288,"statfs":{"total":96636764160,"available":96016678912,"internally_reserved":0,"allocated":375816192,"data_stored":375559288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340600,"kb_used_data":168680,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94031240,"statfs":{"total":96636764160,"available":96287989760,"internally_reserved":0,"allocated":172728320,"data_stored":172423983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692748,"kb_used_data":418172,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93679092,"statfs":{"total":96636764160,"available":95927390208,"internally_reserved":0,"allocated":428208128,"data_stored":427947147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428425216,"data_stored":428389086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172748800,"data_stored":172669314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":375775232,"data_stored":375739102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:59.956 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:59.956 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'7249", 'reported_seq': 7313, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.411988+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:16:55.411988+0000', 'last_peered': '2024-09-11T01:16:55.411988+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:16:55.411988+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 7249, 'log_dups_size': 0, 'ondisk_log_size': 7249, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29691904, 'num_objects': 7249, 'num_object_clones': 0, 'num_object_copies': 14498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7249, 'num_write_kb': 28996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3126}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'7237", 'reported_seq': 7310, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:54.985675+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:54.985675+0000', 'last_peered': '2024-09-11T01:16:54.985675+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:54.985675+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 7237, 'log_dups_size': 0, 'ondisk_log_size': 7237, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29626368, 'num_objects': 7233, 'num_object_clones': 0, 'num_object_copies': 14466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5298, 'num_write_kb': 21192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 3228}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7324", 'reported_seq': 7448, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.361674+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:16:55.361674+0000', 'last_peered': '2024-09-11T01:16:55.361674+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:16:55.361674+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7324, 'log_dups_size': 0, 'ondisk_log_size': 7324, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29979107, 'num_objects': 7320, 'num_object_clones': 0, 'num_object_copies': 14640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7320, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3812, 'num_write_kb': 15248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3205}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'7510", 'reported_seq': 7634, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.397668+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:16:55.397668+0000', 'last_peered': '2024-09-11T01:16:55.397668+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:16:55.397668+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 7510, 'log_dups_size': 0, 'ondisk_log_size': 7510, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30737350, 'num_objects': 7506, 'num_object_clones': 0, 'num_object_copies': 15012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7506, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4263, 'num_write_kb': 17052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3223}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'7320", 'reported_seq': 12446, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:54.975466+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:16:54.975466+0000', 'last_peered': '2024-09-11T01:16:54.975466+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:16:54.975466+0000', 'last_undegraded': '2024-09-11T01:16:54.975466+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 7320, 'log_dups_size': 0, 'ondisk_log_size': 7320, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29954531, 'num_objects': 7314, 'num_object_clones': 0, 'num_object_copies': 14628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1599, 'num_objects_unfound': 0, 'num_objects_dirty': 7314, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5187, 'num_write_kb': 20748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2526, 'num_bytes_recovered': 10338787, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1599}, {'shards': '2', 'objects': 5715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.957 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'7397", 'reported_seq': 7516, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.357749+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:16:55.357749+0000', 'last_peered': '2024-09-11T01:16:55.357749+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:16:55.357749+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 7397, 'log_dups_size': 0, 'ondisk_log_size': 7397, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30265344, 'num_objects': 7389, 'num_object_clones': 0, 'num_object_copies': 14778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3761, 'num_write_kb': 15044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3284}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'7194", 'reported_seq': 7279, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:54.795264+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:54.795264+0000', 'last_peered': '2024-09-11T01:16:54.795264+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:54.795264+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 7194, 'log_dups_size': 0, 'ondisk_log_size': 7194, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29446627, 'num_objects': 7190, 'num_object_clones': 0, 'num_object_copies': 14380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7190, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4963, 'num_write_kb': 19852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 3163}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'7454", 'reported_seq': 7534, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:54.960025+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:54.960025+0000', 'last_peered': '2024-09-11T01:16:54.960025+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:54.960025+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 7454, 'log_dups_size': 0, 'ondisk_log_size': 7454, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30507008, 'num_objects': 7448, 'num_object_clones': 0, 'num_object_copies': 14896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5091, 'num_write_kb': 20364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 3271}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7348", 'reported_seq': 7467, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.389955+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:16:55.389955+0000', 'last_peered': '2024-09-11T01:16:55.389955+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:16:55.389955+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7348, 'log_dups_size': 0, 'ondisk_log_size': 7348, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30089216, 'num_objects': 7346, 'num_object_clones': 0, 'num_object_copies': 14692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4038, 'num_write_kb': 16152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3182}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'7497", 'reported_seq': 7622, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.410027+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:16:55.410027+0000', 'last_peered': '2024-09-11T01:16:55.410027+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:16:55.410027+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 7497, 'log_dups_size': 0, 'ondisk_log_size': 7497, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30674944, 'num_objects': 7489, 'num_object_clones': 0, 'num_object_copies': 14978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7489, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3890, 'num_write_kb': 15560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3297}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.958 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:59.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7394", 'reported_seq': 7517, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.404504+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:16:55.404504+0000', 'last_peered': '2024-09-11T01:16:55.404504+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:16:55.404504+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7394, 'log_dups_size': 0, 'ondisk_log_size': 7394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30269440, 'num_objects': 7390, 'num_object_clones': 0, 'num_object_copies': 14780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3929, 'num_write_kb': 15716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3242}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:59.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'7264", 'reported_seq': 7389, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.402808+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:16:55.402808+0000', 'last_peered': '2024-09-11T01:16:55.402808+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:16:55.402808+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 7264, 'log_dups_size': 0, 'ondisk_log_size': 7264, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29705158, 'num_objects': 7254, 'num_object_clones': 0, 'num_object_copies': 14508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7254, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3921, 'num_write_kb': 15684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3100}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:00.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 144 20 119892 119872 3.25097 1.75 0.0500443 0.0240244 2024-09-11T01:17:00.668 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:00.668 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/236634 objects degraded (19.231%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118317,"data_bytes":484582129,"bytes_used":1670164480,"bytes_avail":288240128000,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":236634,"degraded_ratio":0.19231386867483116,"misplaced_objects":1416,"misplaced_total":236634,"misplaced_ratio":0.0059839245416973049,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":166487,"recovering_keys_per_sec":0,"num_objects_recovered":488,"num_bytes_recovered":1998848,"num_keys_recovered":0,"write_bytes_sec":2005354,"read_op_per_sec":0,"write_op_per_sec":489},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:00.692 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:00.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:01.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:01.272 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:01.302 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-11T01:17:00.452594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484582129,"num_objects":118317,"num_object_clones":0,"num_object_copies":236634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":118317,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":77452,"num_write_kb":309802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2711,"num_bytes_recovered":11092470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118390,"ondisk_log_size":118390,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1631020,"kb_used_data":958460,"kb_used_omap":12,"kb_used_meta":672499,"kb_avail":281484500,"statfs":{"total":289910292480,"available":288240128000,"internally_reserved":0,"allocated":981463040,"data_stored":980640818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":688638983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,7,10,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24076288,"num_objects":5878,"num_object_clones":0,"num_object_copies":11756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-488,"num_objects_unfound":0,"num_objects_dirty":5878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5878,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1998848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006002"},"pg_stats":[{"pgid":"2.f","version":"39'7249","reported_seq":7313,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.411988+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:16:55.411988+0000","last_peered":"2024-09-11T01:16:55.411988+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:16:55.411988+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7249,"log_dups_size":0,"ondisk_log_size":7249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29691904,"num_objects":7249,"num_object_clones":0,"num_object_copies":14498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3126},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7505","reported_seq":7576,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:59.976613+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:59.976613+0000","last_peered":"2024-09-11T01:16:59.976613+0000","last_clean":"2024-09-11T01:16:59.976613+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:59.976613+0000","last_undegraded":"2024-09-11T01:16:59.976613+0000","last_fullsized":"2024-09-11T01:16:59.976613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7505,"log_dups_size":0,"ondisk_log_size":7505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30716870,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5603,"num_write_kb":22412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7400","reported_seq":7473,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.907268+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:59.907268+0000","last_peered":"2024-09-11T01:16:59.907268+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:59.907268+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7400,"log_dups_size":0,"ondisk_log_size":7400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5461,"num_write_kb":21844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7428","reported_seq":7464,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.017823+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:00.017823+0000","last_peered":"2024-09-11T01:17:00.017823+0000","last_clean":"2024-09-11T01:17:00.017823+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:00.017823+0000","last_undegraded":"2024-09-11T01:17:00.017823+0000","last_fullsized":"2024-09-11T01:17:00.017823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7428,"log_dups_size":0,"ondisk_log_size":7428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7428,"num_write_kb":29712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7284","reported_seq":7396,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.374985+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:16:55.374985+0000","last_peered":"2024-09-11T01:16:55.374985+0000","last_clean":"2024-09-11T01:16:55.374985+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:16:55.374985+0000","last_undegraded":"2024-09-11T01:16:55.374985+0000","last_fullsized":"2024-09-11T01:16:55.374985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7284,"log_dups_size":0,"ondisk_log_size":7284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29807075,"num_objects":7278,"num_object_clones":0,"num_object_copies":14556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4082,"num_write_kb":16328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7324","reported_seq":7448,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.361674+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:16:55.361674+0000","last_peered":"2024-09-11T01:16:55.361674+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:16:55.361674+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7324,"log_dups_size":0,"ondisk_log_size":7324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3812,"num_write_kb":15248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3205},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7510","reported_seq":7634,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.397668+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:16:55.397668+0000","last_peered":"2024-09-11T01:16:55.397668+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:16:55.397668+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30737350,"num_objects":7506,"num_object_clones":0,"num_object_copies":15012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7506,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4263,"num_write_kb":17052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3223},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7367","reported_seq":7477,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.334522+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:16:55.334522+0000","last_peered":"2024-09-11T01:16:55.334522+0000","last_clean":"2024-09-11T01:16:55.334522+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:16:55.334522+0000","last_undegraded":"2024-09-11T01:16:55.334522+0000","last_fullsized":"2024-09-11T01:16:55.334522+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30167040,"num_objects":7365,"num_object_clones":0,"num_object_copies":14730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7471","reported_seq":12963,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:00.008994+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:00.008994+0000","last_peered":"2024-09-11T01:17:00.008994+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:00.008994+0000","last_undegraded":"2024-09-11T01:17:00.008994+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7471,"log_dups_size":0,"ondisk_log_size":7471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573027,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5338,"num_write_kb":21352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2709,"num_bytes_recovered":11088355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1416},{"shards":"2","objects":6049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7397","reported_seq":7516,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.357749+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:16:55.357749+0000","last_peered":"2024-09-11T01:16:55.357749+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:16:55.357749+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7397,"log_dups_size":0,"ondisk_log_size":7397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30265344,"num_objects":7389,"num_object_clones":0,"num_object_copies":14778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3761,"num_write_kb":15044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3284},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7342","reported_seq":7427,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.916448+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:59.916448+0000","last_peered":"2024-09-11T01:16:59.916448+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:59.916448+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7342,"log_dups_size":0,"ondisk_log_size":7342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30052835,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5111,"num_write_kb":20444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3311}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7607","reported_seq":7691,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.965440+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:59.965440+0000","last_peered":"2024-09-11T01:16:59.965440+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:59.965440+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5244,"num_write_kb":20976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3424}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7348","reported_seq":7467,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.389955+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:16:55.389955+0000","last_peered":"2024-09-11T01:16:55.389955+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:16:55.389955+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7348,"log_dups_size":0,"ondisk_log_size":7348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30089216,"num_objects":7346,"num_object_clones":0,"num_object_copies":14692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3182},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7497","reported_seq":7622,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.410027+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:16:55.410027+0000","last_peered":"2024-09-11T01:16:55.410027+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:16:55.410027+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7497,"log_dups_size":0,"ondisk_log_size":7497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30674944,"num_objects":7489,"num_object_clones":0,"num_object_copies":14978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3890,"num_write_kb":15560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3297},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7394","reported_seq":7517,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.404504+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:16:55.404504+0000","last_peered":"2024-09-11T01:16:55.404504+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:16:55.404504+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30269440,"num_objects":7390,"num_object_clones":0,"num_object_copies":14780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3242},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7264","reported_seq":7389,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.402808+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:16:55.402808+0000","last_peered":"2024-09-11T01:16:55.402808+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:16:55.402808+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29705158,"num_objects":7254,"num_object_clones":0,"num_object_copies":14508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7254,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3921,"num_write_kb":15684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3100},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484582110,"num_objects":118315,"num_object_clones":0,"num_object_copies":236630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":118315,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":77450,"num_write_kb":309800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2710,"num_bytes_recovered":11092451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":981704704,"data_stored":981552958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118387,"ondisk_log_size":118387,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597672,"kb_used_data":371608,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93774168,"statfs":{"total":96636764160,"available":96024748032,"internally_reserved":0,"allocated":380526592,"data_stored":380269688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789973,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":340600,"kb_used_data":168680,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94031240,"statfs":{"total":96636764160,"available":96287989760,"internally_reserved":0,"allocated":172728320,"data_stored":172423983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692748,"kb_used_data":418172,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93679092,"statfs":{"total":96636764160,"available":95927390208,"internally_reserved":0,"allocated":428208128,"data_stored":427947147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,4,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428425216,"data_stored":428389086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":172748800,"data_stored":172669314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380530688,"data_stored":380494558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:01.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 145 20 120350 120330 3.24089 1.78906 0.0197089 0.0240976 2024-09-11T01:17:02.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 146 20 120867 120847 3.23253 2.01953 0.0763103 0.0241604 2024-09-11T01:17:03.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 147 20 121372 121352 3.22396 1.97266 0.032028 0.0242245 2024-09-11T01:17:04.304 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:04.345 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 148 20 121888 121868 3.21579 2.01562 0.0613961 0.0242844 2024-09-11T01:17:04.889 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:04.892 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:04.913 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-11T01:17:04.453545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490914545,"num_objects":119863,"num_object_clones":0,"num_object_copies":239726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":119863,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":78998,"num_write_kb":315986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2711,"num_bytes_recovered":11092470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119936,"ondisk_log_size":119936,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642140,"kb_used_data":973740,"kb_used_omap":12,"kb_used_meta":668339,"kb_avail":281473380,"statfs":{"total":289910292480,"available":288228741120,"internally_reserved":0,"allocated":997109760,"data_stored":996287538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":684379143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,2,4,8,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20176896,"num_objects":4926,"num_object_clones":0,"num_object_copies":9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4926,"num_write_kb":19704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"2.f","version":"39'7416","reported_seq":7480,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.238436+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:00.238436+0000","last_peered":"2024-09-11T01:17:00.238436+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:00.238436+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":29664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3293},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7505","reported_seq":7576,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:59.976613+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:59.976613+0000","last_peered":"2024-09-11T01:16:59.976613+0000","last_clean":"2024-09-11T01:16:59.976613+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:59.976613+0000","last_undegraded":"2024-09-11T01:16:59.976613+0000","last_fullsized":"2024-09-11T01:16:59.976613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7505,"log_dups_size":0,"ondisk_log_size":7505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30716870,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5603,"num_write_kb":22412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7400","reported_seq":7473,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.907268+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:59.907268+0000","last_peered":"2024-09-11T01:16:59.907268+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:59.907268+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7400,"log_dups_size":0,"ondisk_log_size":7400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5461,"num_write_kb":21844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7428","reported_seq":7464,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.017823+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:00.017823+0000","last_peered":"2024-09-11T01:17:00.017823+0000","last_clean":"2024-09-11T01:17:00.017823+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:00.017823+0000","last_undegraded":"2024-09-11T01:17:00.017823+0000","last_fullsized":"2024-09-11T01:17:00.017823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7428,"log_dups_size":0,"ondisk_log_size":7428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7428,"num_write_kb":29712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7417","reported_seq":7529,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.427024+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:00.427024+0000","last_peered":"2024-09-11T01:17:00.427024+0000","last_clean":"2024-09-11T01:17:00.427024+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:00.427024+0000","last_undegraded":"2024-09-11T01:17:00.427024+0000","last_fullsized":"2024-09-11T01:17:00.427024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7417,"log_dups_size":0,"ondisk_log_size":7417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30351843,"num_objects":7411,"num_object_clones":0,"num_object_copies":14822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4215,"num_write_kb":16860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7475","reported_seq":7600,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.338200+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:00.338200+0000","last_peered":"2024-09-11T01:17:00.338200+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:00.338200+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7475,"log_dups_size":0,"ondisk_log_size":7475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30597603,"num_objects":7471,"num_object_clones":0,"num_object_copies":14942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3356},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7650","reported_seq":7774,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.353651+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:00.353651+0000","last_peered":"2024-09-11T01:17:00.353651+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:00.353651+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7650,"log_dups_size":0,"ondisk_log_size":7650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31310790,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4403,"num_write_kb":17612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3363},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7497","reported_seq":7607,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.352458+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:00.352458+0000","last_peered":"2024-09-11T01:17:00.352458+0000","last_clean":"2024-09-11T01:17:00.352458+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:00.352458+0000","last_undegraded":"2024-09-11T01:17:00.352458+0000","last_fullsized":"2024-09-11T01:17:00.352458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7497,"log_dups_size":0,"ondisk_log_size":7497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":17800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7471","reported_seq":12963,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:00.008994+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:00.008994+0000","last_peered":"2024-09-11T01:17:00.008994+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:00.008994+0000","last_undegraded":"2024-09-11T01:17:00.008994+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7471,"log_dups_size":0,"ondisk_log_size":7471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573027,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5338,"num_write_kb":21352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2709,"num_bytes_recovered":11088355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1416},{"shards":"2","objects":6049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7560","reported_seq":7678,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.431612+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:00.431612+0000","last_peered":"2024-09-11T01:17:00.431612+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:00.431612+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7560,"log_dups_size":0,"ondisk_log_size":7560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30932992,"num_objects":7552,"num_object_clones":0,"num_object_copies":15104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":15696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7342","reported_seq":7427,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.916448+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:59.916448+0000","last_peered":"2024-09-11T01:16:59.916448+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:59.916448+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7342,"log_dups_size":0,"ondisk_log_size":7342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30052835,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5111,"num_write_kb":20444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3311}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7607","reported_seq":7691,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.965440+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:59.965440+0000","last_peered":"2024-09-11T01:16:59.965440+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:59.965440+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5244,"num_write_kb":20976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3424}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7510","reported_seq":7630,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.383441+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:00.383441+0000","last_peered":"2024-09-11T01:17:00.383441+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:00.383441+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3344},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7663","reported_seq":7787,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.326195+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:00.326195+0000","last_peered":"2024-09-11T01:17:00.326195+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:00.326195+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7663,"log_dups_size":0,"ondisk_log_size":7663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4056,"num_write_kb":16224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3463},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7568","reported_seq":7692,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.412552+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:00.412552+0000","last_peered":"2024-09-11T01:17:00.412552+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:00.412552+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7568,"log_dups_size":0,"ondisk_log_size":7568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982144,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3416},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7424","reported_seq":7548,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.388532+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:00.388532+0000","last_peered":"2024-09-11T01:17:00.388532+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:00.388532+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7424,"log_dups_size":0,"ondisk_log_size":7424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30360518,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4081,"num_write_kb":16324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3260},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490914526,"num_objects":119861,"num_object_clones":0,"num_object_copies":239722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":119861,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":78996,"num_write_kb":315984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2710,"num_bytes_recovered":11092451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":997486592,"data_stored":997334846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119933,"ondisk_log_size":119933,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597672,"kb_used_data":371608,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93774168,"statfs":{"total":96636764160,"available":96024748032,"internally_reserved":0,"allocated":380526592,"data_stored":380269688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789974,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352772,"kb_used_data":176692,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94019068,"statfs":{"total":96636764160,"available":96275525632,"internally_reserved":0,"allocated":180932608,"data_stored":180628271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691696,"kb_used_data":425440,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93680144,"statfs":{"total":96636764160,"available":95928467456,"internally_reserved":0,"allocated":435650560,"data_stored":435389579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,6,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435986432,"data_stored":435950302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180969472,"data_stored":180889986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380530688,"data_stored":380494558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:04.914 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:05.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 149 20 122307 122287 3.20519 1.63672 0.0298365 0.024366 2024-09-11T01:17:05.519 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:05.520 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:05.554 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-11T01:17:04.453545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490914545,"num_objects":119863,"num_object_clones":0,"num_object_copies":239726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":119863,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":78998,"num_write_kb":315986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2711,"num_bytes_recovered":11092470,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119936,"ondisk_log_size":119936,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1642140,"kb_used_data":973740,"kb_used_omap":12,"kb_used_meta":668339,"kb_avail":281473380,"statfs":{"total":289910292480,"available":288228741120,"internally_reserved":0,"allocated":997109760,"data_stored":996287538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":684379143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,2,4,8,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20176896,"num_objects":4926,"num_object_clones":0,"num_object_copies":9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4926,"num_write_kb":19704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"2.f","version":"39'7416","reported_seq":7480,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.238436+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:00.238436+0000","last_peered":"2024-09-11T01:17:00.238436+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:00.238436+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7416,"log_dups_size":0,"ondisk_log_size":7416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":29664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3293},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7505","reported_seq":7576,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:16:59.976613+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:16:59.976613+0000","last_peered":"2024-09-11T01:16:59.976613+0000","last_clean":"2024-09-11T01:16:59.976613+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:16:59.976613+0000","last_undegraded":"2024-09-11T01:16:59.976613+0000","last_fullsized":"2024-09-11T01:16:59.976613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7505,"log_dups_size":0,"ondisk_log_size":7505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30716870,"num_objects":7501,"num_object_clones":0,"num_object_copies":15002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7501,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5603,"num_write_kb":22412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7400","reported_seq":7473,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.907268+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:16:59.907268+0000","last_peered":"2024-09-11T01:16:59.907268+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:16:59.907268+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7400,"log_dups_size":0,"ondisk_log_size":7400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5461,"num_write_kb":21844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7428","reported_seq":7464,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.017823+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:00.017823+0000","last_peered":"2024-09-11T01:17:00.017823+0000","last_clean":"2024-09-11T01:17:00.017823+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:00.017823+0000","last_undegraded":"2024-09-11T01:17:00.017823+0000","last_fullsized":"2024-09-11T01:17:00.017823+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7428,"log_dups_size":0,"ondisk_log_size":7428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7428,"num_write_kb":29712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7417","reported_seq":7529,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.427024+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:00.427024+0000","last_peered":"2024-09-11T01:17:00.427024+0000","last_clean":"2024-09-11T01:17:00.427024+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:00.427024+0000","last_undegraded":"2024-09-11T01:17:00.427024+0000","last_fullsized":"2024-09-11T01:17:00.427024+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7417,"log_dups_size":0,"ondisk_log_size":7417,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30351843,"num_objects":7411,"num_object_clones":0,"num_object_copies":14822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4215,"num_write_kb":16860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7475","reported_seq":7600,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.338200+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:00.338200+0000","last_peered":"2024-09-11T01:17:00.338200+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:00.338200+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7475,"log_dups_size":0,"ondisk_log_size":7475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30597603,"num_objects":7471,"num_object_clones":0,"num_object_copies":14942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7471,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3356},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7650","reported_seq":7774,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.353651+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:00.353651+0000","last_peered":"2024-09-11T01:17:00.353651+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:00.353651+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7650,"log_dups_size":0,"ondisk_log_size":7650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31310790,"num_objects":7646,"num_object_clones":0,"num_object_copies":15292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7646,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4403,"num_write_kb":17612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3363},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7497","reported_seq":7607,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.352458+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:00.352458+0000","last_peered":"2024-09-11T01:17:00.352458+0000","last_clean":"2024-09-11T01:17:00.352458+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:00.352458+0000","last_undegraded":"2024-09-11T01:17:00.352458+0000","last_fullsized":"2024-09-11T01:17:00.352458+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7497,"log_dups_size":0,"ondisk_log_size":7497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30699520,"num_objects":7495,"num_object_clones":0,"num_object_copies":14990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4450,"num_write_kb":17800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7471","reported_seq":12963,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:00.008994+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:00.008994+0000","last_peered":"2024-09-11T01:17:00.008994+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:00.008994+0000","last_undegraded":"2024-09-11T01:17:00.008994+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7471,"log_dups_size":0,"ondisk_log_size":7471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573027,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5338,"num_write_kb":21352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2709,"num_bytes_recovered":11088355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1416},{"shards":"2","objects":6049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7560","reported_seq":7678,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.431612+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:00.431612+0000","last_peered":"2024-09-11T01:17:00.431612+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:00.431612+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7560,"log_dups_size":0,"ondisk_log_size":7560,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30932992,"num_objects":7552,"num_object_clones":0,"num_object_copies":15104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":15696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7342","reported_seq":7427,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.916448+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:16:59.916448+0000","last_peered":"2024-09-11T01:16:59.916448+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:16:59.916448+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7342,"log_dups_size":0,"ondisk_log_size":7342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30052835,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5111,"num_write_kb":20444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3311}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7607","reported_seq":7691,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.965440+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:16:59.965440+0000","last_peered":"2024-09-11T01:16:59.965440+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:16:59.965440+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31133696,"num_objects":7601,"num_object_clones":0,"num_object_copies":15202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5244,"num_write_kb":20976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3424}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7510","reported_seq":7630,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.383441+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:00.383441+0000","last_peered":"2024-09-11T01:17:00.383441+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:00.383441+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7510,"log_dups_size":0,"ondisk_log_size":7510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30752768,"num_objects":7508,"num_object_clones":0,"num_object_copies":15016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3344},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7663","reported_seq":7787,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.326195+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:00.326195+0000","last_peered":"2024-09-11T01:17:00.326195+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:00.326195+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7663,"log_dups_size":0,"ondisk_log_size":7663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31354880,"num_objects":7655,"num_object_clones":0,"num_object_copies":15310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4056,"num_write_kb":16224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3463},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7568","reported_seq":7692,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.412552+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:00.412552+0000","last_peered":"2024-09-11T01:17:00.412552+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:00.412552+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7568,"log_dups_size":0,"ondisk_log_size":7568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30982144,"num_objects":7564,"num_object_clones":0,"num_object_copies":15128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3416},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7424","reported_seq":7548,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.388532+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:00.388532+0000","last_peered":"2024-09-11T01:17:00.388532+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:00.388532+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7424,"log_dups_size":0,"ondisk_log_size":7424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30360518,"num_objects":7414,"num_object_clones":0,"num_object_copies":14828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4081,"num_write_kb":16324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3260},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490914526,"num_objects":119861,"num_object_clones":0,"num_object_copies":239722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1416,"num_objects_unfound":0,"num_objects_dirty":119861,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":78996,"num_write_kb":315984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2710,"num_bytes_recovered":11092451,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":997486592,"data_stored":997334846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119933,"ondisk_log_size":119933,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597672,"kb_used_data":371608,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93774168,"statfs":{"total":96636764160,"available":96024748032,"internally_reserved":0,"allocated":380526592,"data_stored":380269688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789974,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352772,"kb_used_data":176692,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94019068,"statfs":{"total":96636764160,"available":96275525632,"internally_reserved":0,"allocated":180932608,"data_stored":180628271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691696,"kb_used_data":425440,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93680144,"statfs":{"total":96636764160,"available":95928467456,"internally_reserved":0,"allocated":435650560,"data_stored":435389579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,6,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":435986432,"data_stored":435950302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180969472,"data_stored":180889986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380530688,"data_stored":380494558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:05.555 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'7416", 'reported_seq': 7480, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.238436+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:00.238436+0000', 'last_peered': '2024-09-11T01:17:00.238436+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:00.238436+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 7416, 'log_dups_size': 0, 'ondisk_log_size': 7416, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30375936, 'num_objects': 7416, 'num_object_clones': 0, 'num_object_copies': 14832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7416, 'num_write_kb': 29664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3293}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'7400", 'reported_seq': 7473, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:59.907268+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:16:59.907268+0000', 'last_peered': '2024-09-11T01:16:59.907268+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:16:59.907268+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 7400, 'log_dups_size': 0, 'ondisk_log_size': 7400, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30294016, 'num_objects': 7396, 'num_object_clones': 0, 'num_object_copies': 14792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7396, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5461, 'num_write_kb': 21844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 3391}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7475", 'reported_seq': 7600, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.338200+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:00.338200+0000', 'last_peered': '2024-09-11T01:17:00.338200+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:00.338200+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7475, 'log_dups_size': 0, 'ondisk_log_size': 7475, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30597603, 'num_objects': 7471, 'num_object_clones': 0, 'num_object_copies': 14942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7471, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3963, 'num_write_kb': 15852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3356}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'7650", 'reported_seq': 7774, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.353651+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:00.353651+0000', 'last_peered': '2024-09-11T01:17:00.353651+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:00.353651+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 7650, 'log_dups_size': 0, 'ondisk_log_size': 7650, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31310790, 'num_objects': 7646, 'num_object_clones': 0, 'num_object_copies': 15292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7646, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4403, 'num_write_kb': 17612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3363}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:05.556 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'7471", 'reported_seq': 12963, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:00.008994+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:17:00.008994+0000', 'last_peered': '2024-09-11T01:17:00.008994+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:17:00.008994+0000', 'last_undegraded': '2024-09-11T01:17:00.008994+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 7471, 'log_dups_size': 0, 'ondisk_log_size': 7471, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30573027, 'num_objects': 7465, 'num_object_clones': 0, 'num_object_copies': 14930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1416, 'num_objects_unfound': 0, 'num_objects_dirty': 7465, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5338, 'num_write_kb': 21352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2709, 'num_bytes_recovered': 11088355, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1416}, {'shards': '2', 'objects': 6049}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'7560", 'reported_seq': 7678, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.431612+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:00.431612+0000', 'last_peered': '2024-09-11T01:17:00.431612+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:00.431612+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 7560, 'log_dups_size': 0, 'ondisk_log_size': 7560, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30932992, 'num_objects': 7552, 'num_object_clones': 0, 'num_object_copies': 15104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3924, 'num_write_kb': 15696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3447}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'7342", 'reported_seq': 7427, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:59.916448+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:16:59.916448+0000', 'last_peered': '2024-09-11T01:16:59.916448+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:16:59.916448+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 7342, 'log_dups_size': 0, 'ondisk_log_size': 7342, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30052835, 'num_objects': 7338, 'num_object_clones': 0, 'num_object_copies': 14676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7338, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5111, 'num_write_kb': 20444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 3311}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'7607", 'reported_seq': 7691, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:59.965440+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:16:59.965440+0000', 'last_peered': '2024-09-11T01:16:59.965440+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:16:59.965440+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 7607, 'log_dups_size': 0, 'ondisk_log_size': 7607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31133696, 'num_objects': 7601, 'num_object_clones': 0, 'num_object_copies': 15202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5244, 'num_write_kb': 20976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 3424}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:05.557 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7510", 'reported_seq': 7630, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.383441+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:00.383441+0000', 'last_peered': '2024-09-11T01:17:00.383441+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:00.383441+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7510, 'log_dups_size': 0, 'ondisk_log_size': 7510, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30752768, 'num_objects': 7508, 'num_object_clones': 0, 'num_object_copies': 15016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4200, 'num_write_kb': 16800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3344}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.558 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:05.558 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'7663", 'reported_seq': 7787, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.326195+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:00.326195+0000', 'last_peered': '2024-09-11T01:17:00.326195+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:00.326195+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 7663, 'log_dups_size': 0, 'ondisk_log_size': 7663, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31354880, 'num_objects': 7655, 'num_object_clones': 0, 'num_object_copies': 15310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4056, 'num_write_kb': 16224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3463}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.558 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:05.558 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7568", 'reported_seq': 7692, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.412552+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:00.412552+0000', 'last_peered': '2024-09-11T01:17:00.412552+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:00.412552+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7568, 'log_dups_size': 0, 'ondisk_log_size': 7568, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30982144, 'num_objects': 7564, 'num_object_clones': 0, 'num_object_copies': 15128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4103, 'num_write_kb': 16412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3416}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.558 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:05.558 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'7424", 'reported_seq': 7548, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.388532+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:00.388532+0000', 'last_peered': '2024-09-11T01:17:00.388532+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:00.388532+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 7424, 'log_dups_size': 0, 'ondisk_log_size': 7424, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30360518, 'num_objects': 7414, 'num_object_clones': 0, 'num_object_copies': 14828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7414, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4081, 'num_write_kb': 16324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3260}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.559 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:06.313 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:06.313 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/239726 objects degraded (18.983%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":226,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119863,"data_bytes":490914545,"bytes_used":1681551360,"bytes_avail":288228741120,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":239726,"degraded_ratio":0.18983339312381639,"misplaced_objects":1416,"misplaced_total":239726,"misplaced_ratio":0.0059067435321992611,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":115000,"recovering_keys_per_sec":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"write_bytes_sec":1680981,"read_op_per_sec":0,"write_op_per_sec":410},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:06.342 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:06.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:06.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 150 20 122757 122737 3.19554 1.75781 0.019084 0.0244397 2024-09-11T01:17:06.916 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:06.916 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:06.948 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-11T01:17:06.454938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":500859633,"num_objects":122291,"num_object_clones":0,"num_object_copies":244582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1246,"num_objects_unfound":0,"num_objects_dirty":122291,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":81426,"num_write_kb":325698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2881,"num_bytes_recovered":11788790,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122364,"ondisk_log_size":122364,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1662804,"kb_used_data":986084,"kb_used_omap":12,"kb_used_meta":676659,"kb_avail":281452716,"statfs":{"total":289910292480,"available":288207581184,"internally_reserved":0,"allocated":1009750016,"data_stored":1008927794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":692898823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,5,5,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30121984,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-507,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7354,"num_write_kb":29416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":507,"num_bytes_recovered":2076672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003981"},"pg_stats":[{"pgid":"2.f","version":"39'7570","reported_seq":7634,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.396732+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:05.396732+0000","last_peered":"2024-09-11T01:17:05.396732+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:05.396732+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7570,"log_dups_size":0,"ondisk_log_size":7570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7661","reported_seq":7732,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.880538+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:04.880538+0000","last_peered":"2024-09-11T01:17:04.880538+0000","last_clean":"2024-09-11T01:17:04.880538+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:04.880538+0000","last_undegraded":"2024-09-11T01:17:04.880538+0000","last_fullsized":"2024-09-11T01:17:04.880538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7661,"log_dups_size":0,"ondisk_log_size":7661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31355846,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5759,"num_write_kb":23036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7543","reported_seq":7616,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.014794+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:05.014794+0000","last_peered":"2024-09-11T01:17:05.014794+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:05.014794+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7543,"log_dups_size":0,"ondisk_log_size":7543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30879744,"num_objects":7539,"num_object_clones":0,"num_object_copies":15078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5604,"num_write_kb":22416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3534}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7588","reported_seq":7624,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.963533+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:04.963533+0000","last_peered":"2024-09-11T01:17:04.963533+0000","last_clean":"2024-09-11T01:17:04.963533+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:04.963533+0000","last_undegraded":"2024-09-11T01:17:04.963533+0000","last_fullsized":"2024-09-11T01:17:04.963533+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7588,"log_dups_size":0,"ondisk_log_size":7588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31080448,"num_objects":7588,"num_object_clones":0,"num_object_copies":15176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7588,"num_write_kb":30352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7563","reported_seq":7675,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.413587+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:05.413587+0000","last_peered":"2024-09-11T01:17:05.413587+0000","last_clean":"2024-09-11T01:17:05.413587+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:05.413587+0000","last_undegraded":"2024-09-11T01:17:05.413587+0000","last_fullsized":"2024-09-11T01:17:05.413587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30949859,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4361,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7615","reported_seq":7737,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.449177+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:05.449177+0000","last_peered":"2024-09-11T01:17:05.449177+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:05.449177+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7615,"log_dups_size":0,"ondisk_log_size":7615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31171043,"num_objects":7611,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7809","reported_seq":7933,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.307297+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:05.307297+0000","last_peered":"2024-09-11T01:17:05.307297+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:05.307297+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962054,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7662","reported_seq":7772,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.347137+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:05.347137+0000","last_peered":"2024-09-11T01:17:05.347137+0000","last_clean":"2024-09-11T01:17:05.347137+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:05.347137+0000","last_undegraded":"2024-09-11T01:17:05.347137+0000","last_fullsized":"2024-09-11T01:17:05.347137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7662,"log_dups_size":0,"ondisk_log_size":7662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":18460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7631","reported_seq":13463,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:04.990514+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:04.990514+0000","last_peered":"2024-09-11T01:17:04.990514+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:04.990514+0000","last_undegraded":"2024-09-11T01:17:04.990514+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7631,"log_dups_size":0,"ondisk_log_size":7631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31228387,"num_objects":7625,"num_object_clones":0,"num_object_copies":15250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1246,"num_objects_unfound":0,"num_objects_dirty":7625,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2879,"num_bytes_recovered":11784675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1246},{"shards":"2","objects":6379}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7725","reported_seq":7844,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.423473+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:05.423473+0000","last_peered":"2024-09-11T01:17:05.423473+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:05.423473+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7725,"log_dups_size":0,"ondisk_log_size":7725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4089,"num_write_kb":16356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3612},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7479","reported_seq":7564,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.901990+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:04.901990+0000","last_peered":"2024-09-11T01:17:04.901990+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:04.901990+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7479,"log_dups_size":0,"ondisk_log_size":7479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613987,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3448}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7763","reported_seq":7847,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.005568+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:05.005568+0000","last_peered":"2024-09-11T01:17:05.005568+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:05.005568+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7763,"log_dups_size":0,"ondisk_log_size":7763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31772672,"num_objects":7757,"num_object_clones":0,"num_object_copies":15514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5400,"num_write_kb":21600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7640","reported_seq":7758,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.469933+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:05.469933+0000","last_peered":"2024-09-11T01:17:05.469933+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:05.469933+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7640,"log_dups_size":0,"ondisk_log_size":7640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3474},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7830","reported_seq":7955,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.472967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:05.472967+0000","last_peered":"2024-09-11T01:17:05.472967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:05.472967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32038912,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4223,"num_write_kb":16892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7713","reported_seq":7837,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.358333+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:05.358333+0000","last_peered":"2024-09-11T01:17:05.358333+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:05.358333+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4248,"num_write_kb":16992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3561},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7569","reported_seq":7694,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.455230+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:05.455230+0000","last_peered":"2024-09-11T01:17:05.455230+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:05.455230+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30954438,"num_objects":7559,"num_object_clones":0,"num_object_copies":15118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3405},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":500859614,"num_objects":122289,"num_object_clones":0,"num_object_copies":244578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1246,"num_objects_unfound":0,"num_objects_dirty":122289,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":81424,"num_write_kb":325696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2880,"num_bytes_recovered":11788771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1009950720,"data_stored":1009798974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122361,"ondisk_log_size":122361,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606804,"kb_used_data":376580,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93765036,"statfs":{"total":96636764160,"available":96015396864,"internally_reserved":0,"allocated":385617920,"data_stored":385361016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789974,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":352772,"kb_used_data":176692,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94019068,"statfs":{"total":96636764160,"available":96275525632,"internally_reserved":0,"allocated":180932608,"data_stored":180628271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703228,"kb_used_data":432812,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93668612,"statfs":{"total":96636764160,"available":95916658688,"internally_reserved":0,"allocated":443199488,"data_stored":442938507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443445248,"data_stored":443409118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180969472,"data_stored":180889986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385536000,"data_stored":385499870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:07.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 151 20 123240 123220 3.18688 1.88672 0.0349529 0.0245063 2024-09-11T01:17:08.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 152 20 123748 123728 3.17897 1.98438 0.0207231 0.0245669 2024-09-11T01:17:09.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 153 20 124263 124243 3.17134 2.01172 0.0346455 0.0246276 2024-09-11T01:17:09.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:10.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 154 19 124745 124726 3.163 1.88672 0.0390801 0.0246922 2024-09-11T01:17:10.517 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:10.520 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:10.542 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-11T01:17:10.456033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":504574705,"num_objects":123198,"num_object_clones":0,"num_object_copies":246396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":123198,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":82333,"num_write_kb":329326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3060,"num_bytes_recovered":12521974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123271,"ondisk_log_size":123271,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1683492,"kb_used_data":998452,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281432028,"statfs":{"total":289910292480,"available":288186396672,"internally_reserved":0,"allocated":1022414848,"data_stored":1021592626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":701418503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,8,13,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-532,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5782,"num_write_kb":23128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":2179072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003908"},"pg_stats":[{"pgid":"2.f","version":"39'7570","reported_seq":7634,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.396732+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:05.396732+0000","last_peered":"2024-09-11T01:17:05.396732+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:05.396732+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7570,"log_dups_size":0,"ondisk_log_size":7570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7808","reported_seq":7879,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.067112+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:10.067112+0000","last_peered":"2024-09-11T01:17:10.067112+0000","last_clean":"2024-09-11T01:17:10.067112+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:10.067112+0000","last_undegraded":"2024-09-11T01:17:10.067112+0000","last_fullsized":"2024-09-11T01:17:10.067112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31957958,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7698","reported_seq":7771,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.054110+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:10.054110+0000","last_peered":"2024-09-11T01:17:10.054110+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:10.054110+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31514624,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5759,"num_write_kb":23036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7742","reported_seq":7778,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.012563+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:10.012563+0000","last_peered":"2024-09-11T01:17:10.012563+0000","last_clean":"2024-09-11T01:17:10.012563+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:10.012563+0000","last_undegraded":"2024-09-11T01:17:10.012563+0000","last_fullsized":"2024-09-11T01:17:10.012563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7742,"num_write_kb":30968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7563","reported_seq":7675,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.413587+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:05.413587+0000","last_peered":"2024-09-11T01:17:05.413587+0000","last_clean":"2024-09-11T01:17:05.413587+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:05.413587+0000","last_undegraded":"2024-09-11T01:17:05.413587+0000","last_fullsized":"2024-09-11T01:17:05.413587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30949859,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4361,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7615","reported_seq":7737,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.449177+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:05.449177+0000","last_peered":"2024-09-11T01:17:05.449177+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:05.449177+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7615,"log_dups_size":0,"ondisk_log_size":7615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31171043,"num_objects":7611,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7809","reported_seq":7933,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.307297+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:05.307297+0000","last_peered":"2024-09-11T01:17:05.307297+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:05.307297+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962054,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7662","reported_seq":7772,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.347137+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:05.347137+0000","last_peered":"2024-09-11T01:17:05.347137+0000","last_clean":"2024-09-11T01:17:05.347137+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:05.347137+0000","last_undegraded":"2024-09-11T01:17:05.347137+0000","last_fullsized":"2024-09-11T01:17:05.347137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7662,"log_dups_size":0,"ondisk_log_size":7662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":18460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7784","reported_seq":13974,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:10.024432+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:10.024432+0000","last_peered":"2024-09-11T01:17:10.024432+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:10.024432+0000","last_undegraded":"2024-09-11T01:17:10.024432+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855075,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3058,"num_bytes_recovered":12517859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1067},{"shards":"2","objects":6711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7725","reported_seq":7844,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.423473+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:05.423473+0000","last_peered":"2024-09-11T01:17:05.423473+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:05.423473+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7725,"log_dups_size":0,"ondisk_log_size":7725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4089,"num_write_kb":16356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3612},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7619","reported_seq":7704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.050329+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:10.050329+0000","last_peered":"2024-09-11T01:17:10.050329+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:10.050329+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7619,"log_dups_size":0,"ondisk_log_size":7619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7921","reported_seq":8004,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.031810+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:10.031810+0000","last_peered":"2024-09-11T01:17:10.031810+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:10.031810+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32419840,"num_objects":7915,"num_object_clones":0,"num_object_copies":15830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7640","reported_seq":7758,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.469933+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:05.469933+0000","last_peered":"2024-09-11T01:17:05.469933+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:05.469933+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7640,"log_dups_size":0,"ondisk_log_size":7640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3474},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7830","reported_seq":7955,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.472967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:05.472967+0000","last_peered":"2024-09-11T01:17:05.472967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:05.472967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32038912,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4223,"num_write_kb":16892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7713","reported_seq":7837,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.358333+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:05.358333+0000","last_peered":"2024-09-11T01:17:05.358333+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:05.358333+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4248,"num_write_kb":16992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3561},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7569","reported_seq":7694,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.455230+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:05.455230+0000","last_peered":"2024-09-11T01:17:05.455230+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:05.455230+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30954438,"num_objects":7559,"num_object_clones":0,"num_object_copies":15118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3405},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":504574686,"num_objects":123196,"num_object_clones":0,"num_object_copies":246392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":123196,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":82331,"num_write_kb":329324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3059,"num_bytes_recovered":12521955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023037440,"data_stored":1022885694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123268,"ondisk_log_size":123268,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615628,"kb_used_data":381244,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93756212,"statfs":{"total":96636764160,"available":96006361088,"internally_reserved":0,"allocated":390393856,"data_stored":390136952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364636,"kb_used_data":184396,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94007204,"statfs":{"total":96636764160,"available":96263376896,"internally_reserved":0,"allocated":188821504,"data_stored":188517167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":184544943},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703228,"kb_used_data":432812,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93668612,"statfs":{"total":96636764160,"available":95916658688,"internally_reserved":0,"allocated":443199488,"data_stored":442938507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443445248,"data_stored":443409118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189067264,"data_stored":188987778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390524928,"data_stored":390488798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:10.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:11.132 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:11.133 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:11.161 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-11T01:17:10.456033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":504574705,"num_objects":123198,"num_object_clones":0,"num_object_copies":246396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":123198,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":82333,"num_write_kb":329326,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3060,"num_bytes_recovered":12521974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123271,"ondisk_log_size":123271,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1683492,"kb_used_data":998452,"kb_used_omap":12,"kb_used_meta":684979,"kb_avail":281432028,"statfs":{"total":289910292480,"available":288186396672,"internally_reserved":0,"allocated":1022414848,"data_stored":1021592626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":701418503},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,8,13,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-532,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5782,"num_write_kb":23128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":532,"num_bytes_recovered":2179072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003908"},"pg_stats":[{"pgid":"2.f","version":"39'7570","reported_seq":7634,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.396732+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:05.396732+0000","last_peered":"2024-09-11T01:17:05.396732+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:05.396732+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7570,"log_dups_size":0,"ondisk_log_size":7570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7808","reported_seq":7879,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.067112+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:10.067112+0000","last_peered":"2024-09-11T01:17:10.067112+0000","last_clean":"2024-09-11T01:17:10.067112+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:10.067112+0000","last_undegraded":"2024-09-11T01:17:10.067112+0000","last_fullsized":"2024-09-11T01:17:10.067112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31957958,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7698","reported_seq":7771,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.054110+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:10.054110+0000","last_peered":"2024-09-11T01:17:10.054110+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:10.054110+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31514624,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5759,"num_write_kb":23036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7742","reported_seq":7778,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.012563+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:10.012563+0000","last_peered":"2024-09-11T01:17:10.012563+0000","last_clean":"2024-09-11T01:17:10.012563+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:10.012563+0000","last_undegraded":"2024-09-11T01:17:10.012563+0000","last_fullsized":"2024-09-11T01:17:10.012563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7742,"num_write_kb":30968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7563","reported_seq":7675,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.413587+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:05.413587+0000","last_peered":"2024-09-11T01:17:05.413587+0000","last_clean":"2024-09-11T01:17:05.413587+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:05.413587+0000","last_undegraded":"2024-09-11T01:17:05.413587+0000","last_fullsized":"2024-09-11T01:17:05.413587+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7563,"log_dups_size":0,"ondisk_log_size":7563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30949859,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4361,"num_write_kb":17444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7615","reported_seq":7737,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.449177+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:05.449177+0000","last_peered":"2024-09-11T01:17:05.449177+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:05.449177+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7615,"log_dups_size":0,"ondisk_log_size":7615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31171043,"num_objects":7611,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3496},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7809","reported_seq":7933,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.307297+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:05.307297+0000","last_peered":"2024-09-11T01:17:05.307297+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:05.307297+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31962054,"num_objects":7805,"num_object_clones":0,"num_object_copies":15610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7805,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3522},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7662","reported_seq":7772,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.347137+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:05.347137+0000","last_peered":"2024-09-11T01:17:05.347137+0000","last_clean":"2024-09-11T01:17:05.347137+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:05.347137+0000","last_undegraded":"2024-09-11T01:17:05.347137+0000","last_fullsized":"2024-09-11T01:17:05.347137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7662,"log_dups_size":0,"ondisk_log_size":7662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":18460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7784","reported_seq":13974,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:10.024432+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:10.024432+0000","last_peered":"2024-09-11T01:17:10.024432+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:10.024432+0000","last_undegraded":"2024-09-11T01:17:10.024432+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855075,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3058,"num_bytes_recovered":12517859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1067},{"shards":"2","objects":6711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7725","reported_seq":7844,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.423473+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:05.423473+0000","last_peered":"2024-09-11T01:17:05.423473+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:05.423473+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7725,"log_dups_size":0,"ondisk_log_size":7725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4089,"num_write_kb":16356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3612},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7619","reported_seq":7704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.050329+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:10.050329+0000","last_peered":"2024-09-11T01:17:10.050329+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:10.050329+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7619,"log_dups_size":0,"ondisk_log_size":7619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7921","reported_seq":8004,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.031810+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:10.031810+0000","last_peered":"2024-09-11T01:17:10.031810+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:10.031810+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32419840,"num_objects":7915,"num_object_clones":0,"num_object_copies":15830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7640","reported_seq":7758,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.469933+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:05.469933+0000","last_peered":"2024-09-11T01:17:05.469933+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:05.469933+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7640,"log_dups_size":0,"ondisk_log_size":7640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31285248,"num_objects":7638,"num_object_clones":0,"num_object_copies":15276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4330,"num_write_kb":17320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3474},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7830","reported_seq":7955,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.472967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:05.472967+0000","last_peered":"2024-09-11T01:17:05.472967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:05.472967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7830,"log_dups_size":0,"ondisk_log_size":7830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32038912,"num_objects":7822,"num_object_clones":0,"num_object_copies":15644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4223,"num_write_kb":16892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7713","reported_seq":7837,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.358333+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:05.358333+0000","last_peered":"2024-09-11T01:17:05.358333+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:05.358333+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7713,"log_dups_size":0,"ondisk_log_size":7713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31576064,"num_objects":7709,"num_object_clones":0,"num_object_copies":15418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4248,"num_write_kb":16992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3561},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7569","reported_seq":7694,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.455230+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:05.455230+0000","last_peered":"2024-09-11T01:17:05.455230+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:05.455230+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7569,"log_dups_size":0,"ondisk_log_size":7569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30954438,"num_objects":7559,"num_object_clones":0,"num_object_copies":15118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4226,"num_write_kb":16904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3405},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":504574686,"num_objects":123196,"num_object_clones":0,"num_object_copies":246392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":123196,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":82331,"num_write_kb":329324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3059,"num_bytes_recovered":12521955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023037440,"data_stored":1022885694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123268,"ondisk_log_size":123268,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615628,"kb_used_data":381244,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93756212,"statfs":{"total":96636764160,"available":96006361088,"internally_reserved":0,"allocated":390393856,"data_stored":390136952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364636,"kb_used_data":184396,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94007204,"statfs":{"total":96636764160,"available":96263376896,"internally_reserved":0,"allocated":188821504,"data_stored":188517167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":184544943},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703228,"kb_used_data":432812,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93668612,"statfs":{"total":96636764160,"available":95916658688,"internally_reserved":0,"allocated":443199488,"data_stored":442938507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443445248,"data_stored":443409118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189067264,"data_stored":188987778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390524928,"data_stored":390488798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'7570", 'reported_seq': 7634, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.396732+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:05.396732+0000', 'last_peered': '2024-09-11T01:17:05.396732+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:05.396732+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 7570, 'log_dups_size': 0, 'ondisk_log_size': 7570, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31006720, 'num_objects': 7570, 'num_object_clones': 0, 'num_object_copies': 15140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7570, 'num_write_kb': 30280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3447}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'7698", 'reported_seq': 7771, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.054110+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:10.054110+0000', 'last_peered': '2024-09-11T01:17:10.054110+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:10.054110+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 7698, 'log_dups_size': 0, 'ondisk_log_size': 7698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31514624, 'num_objects': 7694, 'num_object_clones': 0, 'num_object_copies': 15388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5759, 'num_write_kb': 23036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 3689}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7615", 'reported_seq': 7737, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.449177+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:05.449177+0000', 'last_peered': '2024-09-11T01:17:05.449177+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:05.449177+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7615, 'log_dups_size': 0, 'ondisk_log_size': 7615, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31171043, 'num_objects': 7611, 'num_object_clones': 0, 'num_object_copies': 15222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7611, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4103, 'num_write_kb': 16412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3496}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.163 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'7809", 'reported_seq': 7933, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.307297+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:05.307297+0000', 'last_peered': '2024-09-11T01:17:05.307297+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:05.307297+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 7809, 'log_dups_size': 0, 'ondisk_log_size': 7809, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31962054, 'num_objects': 7805, 'num_object_clones': 0, 'num_object_copies': 15610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7805, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4562, 'num_write_kb': 18248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3522}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'7784", 'reported_seq': 13974, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:10.024432+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:17:10.024432+0000', 'last_peered': '2024-09-11T01:17:10.024432+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:17:10.024432+0000', 'last_undegraded': '2024-09-11T01:17:10.024432+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 7784, 'log_dups_size': 0, 'ondisk_log_size': 7784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31855075, 'num_objects': 7778, 'num_object_clones': 0, 'num_object_copies': 15556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1067, 'num_objects_unfound': 0, 'num_objects_dirty': 7778, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5651, 'num_write_kb': 22604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3058, 'num_bytes_recovered': 12517859, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1067}, {'shards': '2', 'objects': 6711}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'7725", 'reported_seq': 7844, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.423473+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:05.423473+0000', 'last_peered': '2024-09-11T01:17:05.423473+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:05.423473+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 7725, 'log_dups_size': 0, 'ondisk_log_size': 7725, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31608832, 'num_objects': 7717, 'num_object_clones': 0, 'num_object_copies': 15434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4089, 'num_write_kb': 16356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3612}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'7619", 'reported_seq': 7704, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.050329+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:10.050329+0000', 'last_peered': '2024-09-11T01:17:10.050329+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:10.050329+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 7619, 'log_dups_size': 0, 'ondisk_log_size': 7619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31187427, 'num_objects': 7615, 'num_object_clones': 0, 'num_object_copies': 15230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7615, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5388, 'num_write_kb': 21552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 3588}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:11.164 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'7921", 'reported_seq': 8004, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.031810+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:10.031810+0000', 'last_peered': '2024-09-11T01:17:10.031810+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:10.031810+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 7921, 'log_dups_size': 0, 'ondisk_log_size': 7921, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32419840, 'num_objects': 7915, 'num_object_clones': 0, 'num_object_copies': 15830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5558, 'num_write_kb': 22232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 3738}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7640", 'reported_seq': 7758, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.469933+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:05.469933+0000', 'last_peered': '2024-09-11T01:17:05.469933+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:05.469933+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7640, 'log_dups_size': 0, 'ondisk_log_size': 7640, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31285248, 'num_objects': 7638, 'num_object_clones': 0, 'num_object_copies': 15276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4330, 'num_write_kb': 17320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3474}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'7830", 'reported_seq': 7955, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.472967+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:05.472967+0000', 'last_peered': '2024-09-11T01:17:05.472967+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:05.472967+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 7830, 'log_dups_size': 0, 'ondisk_log_size': 7830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32038912, 'num_objects': 7822, 'num_object_clones': 0, 'num_object_copies': 15644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4223, 'num_write_kb': 16892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3630}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7713", 'reported_seq': 7837, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.358333+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:05.358333+0000', 'last_peered': '2024-09-11T01:17:05.358333+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:05.358333+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7713, 'log_dups_size': 0, 'ondisk_log_size': 7713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31576064, 'num_objects': 7709, 'num_object_clones': 0, 'num_object_copies': 15418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4248, 'num_write_kb': 16992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3561}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:11.165 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'7569", 'reported_seq': 7694, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:05.455230+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:05.455230+0000', 'last_peered': '2024-09-11T01:17:05.455230+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:05.455230+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 7569, 'log_dups_size': 0, 'ondisk_log_size': 7569, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30954438, 'num_objects': 7559, 'num_object_clones': 0, 'num_object_copies': 15118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7559, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4226, 'num_write_kb': 16904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3405}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:11.165 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:11.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 155 20 125192 125172 3.15383 1.74219 0.0538802 0.0247644 2024-09-11T01:17:11.869 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:11.869 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/246396 objects degraded (18.469%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":232,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123198,"data_bytes":504574705,"bytes_used":1723895808,"bytes_avail":288186396672,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":246396,"degraded_ratio":0.18469455672981705,"misplaced_objects":1067,"misplaced_total":246396,"misplaced_ratio":0.0043304274420039287,"recovering_objects_per_sec":44,"recovering_bytes_per_sec":181530,"recovering_keys_per_sec":0,"num_objects_recovered":532,"num_bytes_recovered":2179072,"num_keys_recovered":0,"write_bytes_sec":1972946,"read_op_per_sec":0,"write_op_per_sec":481},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:11.896 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:11.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:12.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 156 20 125648 125628 3.14503 1.78125 0.0295038 0.0248346 2024-09-11T01:17:12.519 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:12.531 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:12.539 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-11T01:17:12.456504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510993137,"num_objects":124765,"num_object_clones":0,"num_object_copies":249530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":124765,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":83900,"num_write_kb":335594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3060,"num_bytes_recovered":12521974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124838,"ondisk_log_size":124838,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694884,"kb_used_data":1005684,"kb_used_omap":12,"kb_used_meta":689139,"kb_avail":281420636,"statfs":{"total":289910292480,"available":288174731264,"internally_reserved":0,"allocated":1029820416,"data_stored":1028998194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":705678343},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,8,13,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26411008,"num_objects":6448,"num_object_clones":0,"num_object_copies":12896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-349,"num_objects_unfound":0,"num_objects_dirty":6448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6448,"num_write_kb":25792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":1429504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003910"},"pg_stats":[{"pgid":"2.f","version":"39'7732","reported_seq":7796,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.443849+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:10.443849+0000","last_peered":"2024-09-11T01:17:10.443849+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:10.443849+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7732,"num_write_kb":30928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3609},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7808","reported_seq":7879,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.067112+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:10.067112+0000","last_peered":"2024-09-11T01:17:10.067112+0000","last_clean":"2024-09-11T01:17:10.067112+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:10.067112+0000","last_undegraded":"2024-09-11T01:17:10.067112+0000","last_fullsized":"2024-09-11T01:17:10.067112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31957958,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7698","reported_seq":7771,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.054110+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:10.054110+0000","last_peered":"2024-09-11T01:17:10.054110+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:10.054110+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31514624,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5759,"num_write_kb":23036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7742","reported_seq":7778,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.012563+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:10.012563+0000","last_peered":"2024-09-11T01:17:10.012563+0000","last_clean":"2024-09-11T01:17:10.012563+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:10.012563+0000","last_undegraded":"2024-09-11T01:17:10.012563+0000","last_fullsized":"2024-09-11T01:17:10.012563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7742,"num_write_kb":30968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7726","reported_seq":7838,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.403549+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:10.403549+0000","last_peered":"2024-09-11T01:17:10.403549+0000","last_clean":"2024-09-11T01:17:10.403549+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:10.403549+0000","last_undegraded":"2024-09-11T01:17:10.403549+0000","last_fullsized":"2024-09-11T01:17:10.403549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31617507,"num_objects":7720,"num_object_clones":0,"num_object_copies":15440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4524,"num_write_kb":18096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7762","reported_seq":7887,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.373143+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:10.373143+0000","last_peered":"2024-09-11T01:17:10.373143+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:10.373143+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7762,"log_dups_size":0,"ondisk_log_size":7762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31773155,"num_objects":7758,"num_object_clones":0,"num_object_copies":15516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4250,"num_write_kb":17000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3643},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7980","reported_seq":8104,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.479113+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:10.479113+0000","last_peered":"2024-09-11T01:17:10.479113+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:10.479113+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32662470,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3693},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7821","reported_seq":7930,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.516905+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:10.516905+0000","last_peered":"2024-09-11T01:17:10.516905+0000","last_clean":"2024-09-11T01:17:10.516905+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:10.516905+0000","last_undegraded":"2024-09-11T01:17:10.516905+0000","last_fullsized":"2024-09-11T01:17:10.516905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32026624,"num_objects":7819,"num_object_clones":0,"num_object_copies":15638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7784","reported_seq":13974,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:10.024432+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:10.024432+0000","last_peered":"2024-09-11T01:17:10.024432+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:10.024432+0000","last_undegraded":"2024-09-11T01:17:10.024432+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855075,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3058,"num_bytes_recovered":12517859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1067},{"shards":"2","objects":6711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7888","reported_seq":8006,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.521666+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:10.521666+0000","last_peered":"2024-09-11T01:17:10.521666+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:10.521666+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7888,"log_dups_size":0,"ondisk_log_size":7888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32276480,"num_objects":7880,"num_object_clones":0,"num_object_copies":15760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4252,"num_write_kb":17008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3775},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7619","reported_seq":7704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.050329+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:10.050329+0000","last_peered":"2024-09-11T01:17:10.050329+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:10.050329+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7619,"log_dups_size":0,"ondisk_log_size":7619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7921","reported_seq":8004,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.031810+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:10.031810+0000","last_peered":"2024-09-11T01:17:10.031810+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:10.031810+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32419840,"num_objects":7915,"num_object_clones":0,"num_object_copies":15830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7787","reported_seq":7907,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.517313+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:10.517313+0000","last_peered":"2024-09-11T01:17:10.517313+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:10.517313+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4477,"num_write_kb":17908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7990","reported_seq":8115,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.494127+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:10.494127+0000","last_peered":"2024-09-11T01:17:10.494127+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:10.494127+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7990,"log_dups_size":0,"ondisk_log_size":7990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4383,"num_write_kb":17532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7850","reported_seq":7974,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.537332+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:10.537332+0000","last_peered":"2024-09-11T01:17:10.537332+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:10.537332+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7850,"log_dups_size":0,"ondisk_log_size":7850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32137216,"num_objects":7846,"num_object_clones":0,"num_object_copies":15692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3698},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7727","reported_seq":7849,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.494557+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:10.494557+0000","last_peered":"2024-09-11T01:17:10.494557+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:10.494557+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7727,"log_dups_size":0,"ondisk_log_size":7727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31601606,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4384,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3563},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510993118,"num_objects":124763,"num_object_clones":0,"num_object_copies":249526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":124763,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":83898,"num_write_kb":335592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3059,"num_bytes_recovered":12521955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1030664192,"data_stored":1030512446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124835,"ondisk_log_size":124835,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615628,"kb_used_data":381244,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93756212,"statfs":{"total":96636764160,"available":96006361088,"internally_reserved":0,"allocated":390393856,"data_stored":390136952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789975,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364636,"kb_used_data":184396,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":94007204,"statfs":{"total":96636764160,"available":96263376896,"internally_reserved":0,"allocated":188821504,"data_stored":188517167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":184544943},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714620,"kb_used_data":440044,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93657220,"statfs":{"total":96636764160,"available":95904993280,"internally_reserved":0,"allocated":450605056,"data_stored":450344075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451072000,"data_stored":451035870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189067264,"data_stored":188987778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390524928,"data_stored":390488798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:13.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 157 20 126132 126112 3.13704 1.89062 0.0243378 0.0248946 2024-09-11T01:17:14.346 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 158 20 126645 126625 3.12987 2.00391 0.0344429 0.0249525 2024-09-11T01:17:15.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 159 20 127155 127135 3.12271 1.99219 0.0248681 0.0250105 2024-09-11T01:17:15.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:16.101 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:16.102 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:16.134 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-11T01:17:14.457045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510993137,"num_objects":124765,"num_object_clones":0,"num_object_copies":249530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":124765,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":83900,"num_write_kb":335594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3060,"num_bytes_recovered":12521974,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124838,"ondisk_log_size":124838,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1694488,"kb_used_data":1013608,"kb_used_omap":12,"kb_used_meta":680819,"kb_avail":281421032,"statfs":{"total":289910292480,"available":288175136768,"internally_reserved":0,"allocated":1037934592,"data_stored":1037112370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":697158663},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,7,10,9,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20078592,"num_objects":4902,"num_object_clones":0,"num_object_copies":9804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-349,"num_objects_unfound":0,"num_objects_dirty":4902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4902,"num_write_kb":19608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":1429504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003999"},"pg_stats":[{"pgid":"2.f","version":"39'7732","reported_seq":7796,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.443849+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:10.443849+0000","last_peered":"2024-09-11T01:17:10.443849+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:10.443849+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7732,"log_dups_size":0,"ondisk_log_size":7732,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7732,"num_write_kb":30928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3609},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7808","reported_seq":7879,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.067112+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:10.067112+0000","last_peered":"2024-09-11T01:17:10.067112+0000","last_clean":"2024-09-11T01:17:10.067112+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:10.067112+0000","last_undegraded":"2024-09-11T01:17:10.067112+0000","last_fullsized":"2024-09-11T01:17:10.067112+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7808,"log_dups_size":0,"ondisk_log_size":7808,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31957958,"num_objects":7804,"num_object_clones":0,"num_object_copies":15608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7804,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5906,"num_write_kb":23624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7698","reported_seq":7771,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.054110+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:10.054110+0000","last_peered":"2024-09-11T01:17:10.054110+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:10.054110+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31514624,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5759,"num_write_kb":23036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7742","reported_seq":7778,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.012563+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:10.012563+0000","last_peered":"2024-09-11T01:17:10.012563+0000","last_clean":"2024-09-11T01:17:10.012563+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:10.012563+0000","last_undegraded":"2024-09-11T01:17:10.012563+0000","last_fullsized":"2024-09-11T01:17:10.012563+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7742,"num_write_kb":30968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7726","reported_seq":7838,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.403549+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:10.403549+0000","last_peered":"2024-09-11T01:17:10.403549+0000","last_clean":"2024-09-11T01:17:10.403549+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:10.403549+0000","last_undegraded":"2024-09-11T01:17:10.403549+0000","last_fullsized":"2024-09-11T01:17:10.403549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31617507,"num_objects":7720,"num_object_clones":0,"num_object_copies":15440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4524,"num_write_kb":18096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7762","reported_seq":7887,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.373143+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:10.373143+0000","last_peered":"2024-09-11T01:17:10.373143+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:10.373143+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7762,"log_dups_size":0,"ondisk_log_size":7762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31773155,"num_objects":7758,"num_object_clones":0,"num_object_copies":15516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4250,"num_write_kb":17000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3643},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'7980","reported_seq":8104,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.479113+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:10.479113+0000","last_peered":"2024-09-11T01:17:10.479113+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:10.479113+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32662470,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4733,"num_write_kb":18932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3693},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7821","reported_seq":7930,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.516905+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:10.516905+0000","last_peered":"2024-09-11T01:17:10.516905+0000","last_clean":"2024-09-11T01:17:10.516905+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:10.516905+0000","last_undegraded":"2024-09-11T01:17:10.516905+0000","last_fullsized":"2024-09-11T01:17:10.516905+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7821,"log_dups_size":0,"ondisk_log_size":7821,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32026624,"num_objects":7819,"num_object_clones":0,"num_object_copies":15638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4774,"num_write_kb":19096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7784","reported_seq":13974,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:10.024432+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:10.024432+0000","last_peered":"2024-09-11T01:17:10.024432+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:10.024432+0000","last_undegraded":"2024-09-11T01:17:10.024432+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7784,"log_dups_size":0,"ondisk_log_size":7784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31855075,"num_objects":7778,"num_object_clones":0,"num_object_copies":15556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":7778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5651,"num_write_kb":22604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3058,"num_bytes_recovered":12517859,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1067},{"shards":"2","objects":6711}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'7888","reported_seq":8006,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.521666+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:10.521666+0000","last_peered":"2024-09-11T01:17:10.521666+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:10.521666+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":7888,"log_dups_size":0,"ondisk_log_size":7888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32276480,"num_objects":7880,"num_object_clones":0,"num_object_copies":15760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4252,"num_write_kb":17008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3775},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7619","reported_seq":7704,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.050329+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:10.050329+0000","last_peered":"2024-09-11T01:17:10.050329+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:10.050329+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7619,"log_dups_size":0,"ondisk_log_size":7619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5388,"num_write_kb":21552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'7921","reported_seq":8004,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.031810+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:10.031810+0000","last_peered":"2024-09-11T01:17:10.031810+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:10.031810+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":7921,"log_dups_size":0,"ondisk_log_size":7921,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32419840,"num_objects":7915,"num_object_clones":0,"num_object_copies":15830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7787","reported_seq":7907,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.517313+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:10.517313+0000","last_peered":"2024-09-11T01:17:10.517313+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:10.517313+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7787,"log_dups_size":0,"ondisk_log_size":7787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4477,"num_write_kb":17908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'7990","reported_seq":8115,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.494127+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:10.494127+0000","last_peered":"2024-09-11T01:17:10.494127+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:10.494127+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":7990,"log_dups_size":0,"ondisk_log_size":7990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4383,"num_write_kb":17532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7850","reported_seq":7974,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.537332+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:10.537332+0000","last_peered":"2024-09-11T01:17:10.537332+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:10.537332+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7850,"log_dups_size":0,"ondisk_log_size":7850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32137216,"num_objects":7846,"num_object_clones":0,"num_object_copies":15692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3698},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7727","reported_seq":7849,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.494557+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:10.494557+0000","last_peered":"2024-09-11T01:17:10.494557+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:10.494557+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7727,"log_dups_size":0,"ondisk_log_size":7727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31601606,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4384,"num_write_kb":17536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3563},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510993118,"num_objects":124763,"num_object_clones":0,"num_object_copies":249526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":1067,"num_objects_unfound":0,"num_objects_dirty":124763,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":83898,"num_write_kb":335592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3059,"num_bytes_recovered":12521955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1038729216,"data_stored":1038577470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124835,"ondisk_log_size":124835,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615628,"kb_used_data":381244,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93756212,"statfs":{"total":96636764160,"available":96006361088,"internally_reserved":0,"allocated":390393856,"data_stored":390136952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364240,"kb_used_data":192320,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94007600,"statfs":{"total":96636764160,"available":96263782400,"internally_reserved":0,"allocated":196935680,"data_stored":196631343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714620,"kb_used_data":440044,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93657220,"statfs":{"total":96636764160,"available":95904993280,"internally_reserved":0,"allocated":450605056,"data_stored":450344075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,3,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":451072000,"data_stored":451035870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197132288,"data_stored":197052802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390524928,"data_stored":390488798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:16.136 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:16.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:17:16.346223+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.025079 2024-09-11T01:17:16.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:17:16.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 160 20 127615 127595 3.11443 1.79688 0.0892559 0.025079 2024-09-11T01:17:16.724 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:16.727 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:16.760 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-11T01:17:16.457499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520917745,"num_objects":127188,"num_object_clones":0,"num_object_copies":254376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":892,"num_objects_unfound":0,"num_objects_dirty":127188,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":86323,"num_write_kb":345286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3235,"num_bytes_recovered":13238774,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127261,"ondisk_log_size":127261,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1698208,"kb_used_data":1025648,"kb_used_omap":12,"kb_used_meta":672499,"kb_avail":281417312,"statfs":{"total":289910292480,"available":288171327488,"internally_reserved":0,"allocated":1050263552,"data_stored":1049441330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":688638983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,9,9,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30003200,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-524,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":29300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":2146304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003953"},"pg_stats":[{"pgid":"2.f","version":"39'7890","reported_seq":7954,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.574186+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:15.574186+0000","last_peered":"2024-09-11T01:17:15.574186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:15.574186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":31560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7962","reported_seq":8033,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.053482+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:15.053482+0000","last_peered":"2024-09-11T01:17:15.053482+0000","last_clean":"2024-09-11T01:17:15.053482+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:15.053482+0000","last_undegraded":"2024-09-11T01:17:15.053482+0000","last_fullsized":"2024-09-11T01:17:15.053482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32588742,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6060,"num_write_kb":24240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7838","reported_seq":7910,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.005307+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:15.005307+0000","last_peered":"2024-09-11T01:17:15.005307+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:15.005307+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7838,"log_dups_size":0,"ondisk_log_size":7838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3829}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7909","reported_seq":7945,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.002515+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:15.002515+0000","last_peered":"2024-09-11T01:17:15.002515+0000","last_clean":"2024-09-11T01:17:15.002515+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:15.002515+0000","last_undegraded":"2024-09-11T01:17:15.002515+0000","last_fullsized":"2024-09-11T01:17:15.002515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7909,"log_dups_size":0,"ondisk_log_size":7909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32395264,"num_objects":7909,"num_object_clones":0,"num_object_copies":15818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7909,"num_write_kb":31636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7876","reported_seq":7988,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.569244+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:15.569244+0000","last_peered":"2024-09-11T01:17:15.569244+0000","last_clean":"2024-09-11T01:17:15.569244+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:15.569244+0000","last_undegraded":"2024-09-11T01:17:15.569244+0000","last_fullsized":"2024-09-11T01:17:15.569244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7876,"log_dups_size":0,"ondisk_log_size":7876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231907,"num_objects":7870,"num_object_clones":0,"num_object_copies":15740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4674,"num_write_kb":18696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7924","reported_seq":8049,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.538013+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:15.538013+0000","last_peered":"2024-09-11T01:17:15.538013+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:15.538013+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436707,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4412,"num_write_kb":17648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8126","reported_seq":8250,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.365592+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:15.365592+0000","last_peered":"2024-09-11T01:17:15.365592+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:15.365592+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8126,"log_dups_size":0,"ondisk_log_size":8126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33260486,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3839},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7975","reported_seq":8085,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.544695+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:15.544695+0000","last_peered":"2024-09-11T01:17:15.544695+0000","last_clean":"2024-09-11T01:17:15.544695+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:15.544695+0000","last_undegraded":"2024-09-11T01:17:15.544695+0000","last_fullsized":"2024-09-11T01:17:15.544695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4928,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7948","reported_seq":14487,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:15.087386+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:15.087386+0000","last_peered":"2024-09-11T01:17:15.087386+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:15.087386+0000","last_undegraded":"2024-09-11T01:17:15.087386+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7948,"log_dups_size":0,"ondisk_log_size":7948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526819,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":892,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5815,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3233,"num_bytes_recovered":13234659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":892},{"shards":"2","objects":7050}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8046","reported_seq":8164,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.558766+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:15.558766+0000","last_peered":"2024-09-11T01:17:15.558766+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:15.558766+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7757","reported_seq":7840,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.063648+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:15.063648+0000","last_peered":"2024-09-11T01:17:15.063648+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:15.063648+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7757,"log_dups_size":0,"ondisk_log_size":7757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31752675,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5526,"num_write_kb":22104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8045","reported_seq":8129,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.061650+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:15.061650+0000","last_peered":"2024-09-11T01:17:15.061650+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:15.061650+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8045,"log_dups_size":0,"ondisk_log_size":8045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32927744,"num_objects":8039,"num_object_clones":0,"num_object_copies":16078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3862}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7954","reported_seq":8075,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.511983+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:15.511983+0000","last_peered":"2024-09-11T01:17:15.511983+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:15.511983+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7954,"log_dups_size":0,"ondisk_log_size":7954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8121","reported_seq":8246,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.579196+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:15.579196+0000","last_peered":"2024-09-11T01:17:15.579196+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:15.579196+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33230848,"num_objects":8113,"num_object_clones":0,"num_object_copies":16226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7999","reported_seq":8123,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.592238+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:15.592238+0000","last_peered":"2024-09-11T01:17:15.592238+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:15.592238+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7999,"log_dups_size":0,"ondisk_log_size":7999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32747520,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4534,"num_write_kb":18136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7888","reported_seq":8014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.477681+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:15.477681+0000","last_peered":"2024-09-11T01:17:15.477681+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:15.477681+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7888,"log_dups_size":0,"ondisk_log_size":7888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32261062,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3724},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520917726,"num_objects":127186,"num_object_clones":0,"num_object_copies":254372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":892,"num_objects_unfound":0,"num_objects_dirty":127186,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":86321,"num_write_kb":345284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3234,"num_bytes_recovered":13238755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051242496,"data_stored":1051090750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127258,"ondisk_log_size":127258,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624596,"kb_used_data":386052,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93747244,"statfs":{"total":96636764160,"available":95997177856,"internally_reserved":0,"allocated":395317248,"data_stored":395060344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364240,"kb_used_data":192320,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94007600,"statfs":{"total":96636764160,"available":96263782400,"internally_reserved":0,"allocated":196935680,"data_stored":196631343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709372,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93662468,"statfs":{"total":96636764160,"available":95910367232,"internally_reserved":0,"allocated":458010624,"data_stored":457749643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458698752,"data_stored":458662622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197132288,"data_stored":197052802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395411456,"data_stored":395375326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:16.762 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:16.762 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'7890", 'reported_seq': 7954, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.574186+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:15.574186+0000', 'last_peered': '2024-09-11T01:17:15.574186+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:15.574186+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 7890, 'log_dups_size': 0, 'ondisk_log_size': 7890, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32317440, 'num_objects': 7890, 'num_object_clones': 0, 'num_object_copies': 15780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7890, 'num_write_kb': 31560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3767}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.762 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'7838", 'reported_seq': 7910, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.005307+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:15.005307+0000', 'last_peered': '2024-09-11T01:17:15.005307+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:15.005307+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 7838, 'log_dups_size': 0, 'ondisk_log_size': 7838, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32088064, 'num_objects': 7834, 'num_object_clones': 0, 'num_object_copies': 15668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7834, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5899, 'num_write_kb': 23596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 3829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7924", 'reported_seq': 8049, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.538013+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:15.538013+0000', 'last_peered': '2024-09-11T01:17:15.538013+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:15.538013+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7924, 'log_dups_size': 0, 'ondisk_log_size': 7924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32436707, 'num_objects': 7920, 'num_object_clones': 0, 'num_object_copies': 15840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7920, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4412, 'num_write_kb': 17648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3805}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'8126", 'reported_seq': 8250, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.365592+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:15.365592+0000', 'last_peered': '2024-09-11T01:17:15.365592+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:15.365592+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 8126, 'log_dups_size': 0, 'ondisk_log_size': 8126, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33260486, 'num_objects': 8122, 'num_object_clones': 0, 'num_object_copies': 16244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8122, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4879, 'num_write_kb': 19516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3839}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'7948", 'reported_seq': 14487, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:15.087386+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:17:15.087386+0000', 'last_peered': '2024-09-11T01:17:15.087386+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:17:15.087386+0000', 'last_undegraded': '2024-09-11T01:17:15.087386+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 7948, 'log_dups_size': 0, 'ondisk_log_size': 7948, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32526819, 'num_objects': 7942, 'num_object_clones': 0, 'num_object_copies': 15884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 892, 'num_objects_unfound': 0, 'num_objects_dirty': 7942, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5815, 'num_write_kb': 23260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3233, 'num_bytes_recovered': 13234659, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 892}, {'shards': '2', 'objects': 7050}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:16.763 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'8046", 'reported_seq': 8164, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.558766+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:15.558766+0000', 'last_peered': '2024-09-11T01:17:15.558766+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:15.558766+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8046, 'log_dups_size': 0, 'ondisk_log_size': 8046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32923648, 'num_objects': 8038, 'num_object_clones': 0, 'num_object_copies': 16076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4410, 'num_write_kb': 17640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3933}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'7757", 'reported_seq': 7840, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.063648+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:15.063648+0000', 'last_peered': '2024-09-11T01:17:15.063648+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:15.063648+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 7757, 'log_dups_size': 0, 'ondisk_log_size': 7757, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31752675, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5526, 'num_write_kb': 22104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 3726}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'8045", 'reported_seq': 8129, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.061650+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:15.061650+0000', 'last_peered': '2024-09-11T01:17:15.061650+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:15.061650+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8045, 'log_dups_size': 0, 'ondisk_log_size': 8045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32927744, 'num_objects': 8039, 'num_object_clones': 0, 'num_object_copies': 16078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5682, 'num_write_kb': 22728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 3862}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7954", 'reported_seq': 8075, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.511983+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:15.511983+0000', 'last_peered': '2024-09-11T01:17:15.511983+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:15.511983+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7954, 'log_dups_size': 0, 'ondisk_log_size': 7954, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32571392, 'num_objects': 7952, 'num_object_clones': 0, 'num_object_copies': 15904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4644, 'num_write_kb': 18576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3788}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'8121", 'reported_seq': 8246, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.579196+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:15.579196+0000', 'last_peered': '2024-09-11T01:17:15.579196+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:15.579196+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 8121, 'log_dups_size': 0, 'ondisk_log_size': 8121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33230848, 'num_objects': 8113, 'num_object_clones': 0, 'num_object_copies': 16226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4514, 'num_write_kb': 18056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3921}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:16.764 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7999", 'reported_seq': 8123, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.592238+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:15.592238+0000', 'last_peered': '2024-09-11T01:17:15.592238+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:15.592238+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7999, 'log_dups_size': 0, 'ondisk_log_size': 7999, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32747520, 'num_objects': 7995, 'num_object_clones': 0, 'num_object_copies': 15990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7995, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4534, 'num_write_kb': 18136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.765 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:16.765 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'7888", 'reported_seq': 8014, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.477681+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:15.477681+0000', 'last_peered': '2024-09-11T01:17:15.477681+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:15.477681+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 7888, 'log_dups_size': 0, 'ondisk_log_size': 7888, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32261062, 'num_objects': 7878, 'num_object_clones': 0, 'num_object_copies': 15756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7878, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4545, 'num_write_kb': 18180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3724}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:16.765 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:17.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 161 20 128051 128031 3.10566 1.70312 0.0245976 0.0251467 2024-09-11T01:17:17.472 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:17.472 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/254376 objects degraded (17.890%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":237,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":127188,"data_bytes":520917745,"bytes_used":1738964992,"bytes_avail":288171327488,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":254376,"degraded_ratio":0.17890052520678051,"misplaced_objects":892,"misplaced_total":254376,"misplaced_ratio":0.0035066201213951004,"recovering_objects_per_sec":43,"recovering_bytes_per_sec":178799,"recovering_keys_per_sec":0,"num_objects_recovered":524,"num_bytes_recovered":2146304,"num_keys_recovered":0,"write_bytes_sec":2499443,"read_op_per_sec":0,"write_op_per_sec":610},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:17.501 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:17.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:18.068 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:18.070 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:18.100 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":151,"stamp":"2024-09-11T01:17:16.457499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520917745,"num_objects":127188,"num_object_clones":0,"num_object_copies":254376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":892,"num_objects_unfound":0,"num_objects_dirty":127188,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":86323,"num_write_kb":345286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3235,"num_bytes_recovered":13238774,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127261,"ondisk_log_size":127261,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1698208,"kb_used_data":1025648,"kb_used_omap":12,"kb_used_meta":672499,"kb_avail":281417312,"statfs":{"total":289910292480,"available":288171327488,"internally_reserved":0,"allocated":1050263552,"data_stored":1049441330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":688638983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,9,9,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30003200,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-524,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7325,"num_write_kb":29300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":2146304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003953"},"pg_stats":[{"pgid":"2.f","version":"39'7890","reported_seq":7954,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.574186+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:15.574186+0000","last_peered":"2024-09-11T01:17:15.574186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:15.574186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":31560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'7962","reported_seq":8033,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.053482+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:15.053482+0000","last_peered":"2024-09-11T01:17:15.053482+0000","last_clean":"2024-09-11T01:17:15.053482+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:15.053482+0000","last_undegraded":"2024-09-11T01:17:15.053482+0000","last_fullsized":"2024-09-11T01:17:15.053482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":7962,"log_dups_size":0,"ondisk_log_size":7962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32588742,"num_objects":7958,"num_object_clones":0,"num_object_copies":15916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6060,"num_write_kb":24240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7838","reported_seq":7910,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.005307+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:15.005307+0000","last_peered":"2024-09-11T01:17:15.005307+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:15.005307+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7838,"log_dups_size":0,"ondisk_log_size":7838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32088064,"num_objects":7834,"num_object_clones":0,"num_object_copies":15668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5899,"num_write_kb":23596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3829}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'7909","reported_seq":7945,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.002515+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:15.002515+0000","last_peered":"2024-09-11T01:17:15.002515+0000","last_clean":"2024-09-11T01:17:15.002515+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:15.002515+0000","last_undegraded":"2024-09-11T01:17:15.002515+0000","last_fullsized":"2024-09-11T01:17:15.002515+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7909,"log_dups_size":0,"ondisk_log_size":7909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32395264,"num_objects":7909,"num_object_clones":0,"num_object_copies":15818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7909,"num_write_kb":31636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7876","reported_seq":7988,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.569244+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:15.569244+0000","last_peered":"2024-09-11T01:17:15.569244+0000","last_clean":"2024-09-11T01:17:15.569244+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:15.569244+0000","last_undegraded":"2024-09-11T01:17:15.569244+0000","last_fullsized":"2024-09-11T01:17:15.569244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7876,"log_dups_size":0,"ondisk_log_size":7876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231907,"num_objects":7870,"num_object_clones":0,"num_object_copies":15740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4674,"num_write_kb":18696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7924","reported_seq":8049,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.538013+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:15.538013+0000","last_peered":"2024-09-11T01:17:15.538013+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:15.538013+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436707,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4412,"num_write_kb":17648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8126","reported_seq":8250,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.365592+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:15.365592+0000","last_peered":"2024-09-11T01:17:15.365592+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:15.365592+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8126,"log_dups_size":0,"ondisk_log_size":8126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33260486,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3839},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7975","reported_seq":8085,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.544695+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:15.544695+0000","last_peered":"2024-09-11T01:17:15.544695+0000","last_clean":"2024-09-11T01:17:15.544695+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:15.544695+0000","last_undegraded":"2024-09-11T01:17:15.544695+0000","last_fullsized":"2024-09-11T01:17:15.544695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4928,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.992931+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.992931+0000","last_peered":"2024-09-11T01:15:19.992931+0000","last_clean":"2024-09-11T01:15:19.992931+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.992931+0000","last_undegraded":"2024-09-11T01:15:19.992931+0000","last_fullsized":"2024-09-11T01:15:19.992931+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'7948","reported_seq":14487,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:15.087386+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:15.087386+0000","last_peered":"2024-09-11T01:17:15.087386+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:15.087386+0000","last_undegraded":"2024-09-11T01:17:15.087386+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":7948,"log_dups_size":0,"ondisk_log_size":7948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526819,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":892,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5815,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3233,"num_bytes_recovered":13234659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":892},{"shards":"2","objects":7050}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8046","reported_seq":8164,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.558766+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:15.558766+0000","last_peered":"2024-09-11T01:17:15.558766+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:15.558766+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7757","reported_seq":7840,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.063648+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:15.063648+0000","last_peered":"2024-09-11T01:17:15.063648+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:15.063648+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7757,"log_dups_size":0,"ondisk_log_size":7757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31752675,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5526,"num_write_kb":22104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8045","reported_seq":8129,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.061650+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:15.061650+0000","last_peered":"2024-09-11T01:17:15.061650+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:15.061650+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8045,"log_dups_size":0,"ondisk_log_size":8045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32927744,"num_objects":8039,"num_object_clones":0,"num_object_copies":16078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":3862}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7954","reported_seq":8075,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.511983+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:15.511983+0000","last_peered":"2024-09-11T01:17:15.511983+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:15.511983+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7954,"log_dups_size":0,"ondisk_log_size":7954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8121","reported_seq":8246,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.579196+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:15.579196+0000","last_peered":"2024-09-11T01:17:15.579196+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:15.579196+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33230848,"num_objects":8113,"num_object_clones":0,"num_object_copies":16226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7999","reported_seq":8123,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.592238+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:15.592238+0000","last_peered":"2024-09-11T01:17:15.592238+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:15.592238+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7999,"log_dups_size":0,"ondisk_log_size":7999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32747520,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4534,"num_write_kb":18136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7888","reported_seq":8014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.477681+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:15.477681+0000","last_peered":"2024-09-11T01:17:15.477681+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:15.477681+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7888,"log_dups_size":0,"ondisk_log_size":7888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32261062,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3724},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520917726,"num_objects":127186,"num_object_clones":0,"num_object_copies":254372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":892,"num_objects_unfound":0,"num_objects_dirty":127186,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":86321,"num_write_kb":345284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3234,"num_bytes_recovered":13238755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051242496,"data_stored":1051090750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127258,"ondisk_log_size":127258,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":624596,"kb_used_data":386052,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93747244,"statfs":{"total":96636764160,"available":95997177856,"internally_reserved":0,"allocated":395317248,"data_stored":395060344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789976,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":364240,"kb_used_data":192320,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94007600,"statfs":{"total":96636764160,"available":96263782400,"internally_reserved":0,"allocated":196935680,"data_stored":196631343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709372,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93662468,"statfs":{"total":96636764160,"available":95910367232,"internally_reserved":0,"allocated":458010624,"data_stored":457749643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458698752,"data_stored":458662622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":197132288,"data_stored":197052802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395411456,"data_stored":395375326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:18.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 162 20 128510 128490 3.09756 1.79297 0.0295728 0.0252137 2024-09-11T01:17:19.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 163 20 129014 128994 3.09063 1.96875 0.130076 0.0252691 2024-09-11T01:17:20.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 164 20 129527 129507 3.08401 2.00391 0.0286117 0.0253218 2024-09-11T01:17:21.103 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:21.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 165 20 130007 129987 3.07668 1.875 0.045169 0.0253846 2024-09-11T01:17:21.695 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:21.698 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:21.720 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-11T01:17:20.462313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524661489,"num_objects":128102,"num_object_clones":0,"num_object_copies":256204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":128102,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":87237,"num_write_kb":348942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":14000630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128175,"ondisk_log_size":128175,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1702584,"kb_used_data":1038344,"kb_used_omap":12,"kb_used_meta":664179,"kb_avail":281412936,"statfs":{"total":289910292480,"available":288166846464,"internally_reserved":0,"allocated":1063264256,"data_stored":1062442034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":680119303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,11,6,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-540,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":23244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":2211840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006751"},"pg_stats":[{"pgid":"2.f","version":"39'7890","reported_seq":7954,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.574186+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:15.574186+0000","last_peered":"2024-09-11T01:17:15.574186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:15.574186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":31560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8116","reported_seq":8187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.044266+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:20.044266+0000","last_peered":"2024-09-11T01:17:20.044266+0000","last_clean":"2024-09-11T01:17:20.044266+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:20.044266+0000","last_undegraded":"2024-09-11T01:17:20.044266+0000","last_fullsized":"2024-09-11T01:17:20.044266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33219526,"num_objects":8112,"num_object_clones":0,"num_object_copies":16224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7989","reported_seq":8063,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.039005+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:20.039005+0000","last_peered":"2024-09-11T01:17:20.039005+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:20.039005+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7989,"log_dups_size":0,"ondisk_log_size":7989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32706560,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6050,"num_write_kb":24200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3980}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8054","reported_seq":8087,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.086460+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:20.086460+0000","last_peered":"2024-09-11T01:17:20.086460+0000","last_clean":"2024-09-11T01:17:20.086460+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:20.086460+0000","last_undegraded":"2024-09-11T01:17:20.086460+0000","last_fullsized":"2024-09-11T01:17:20.086460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8054,"log_dups_size":0,"ondisk_log_size":8054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989184,"num_objects":8054,"num_object_clones":0,"num_object_copies":16108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8054,"num_write_kb":32216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7876","reported_seq":7988,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.569244+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:15.569244+0000","last_peered":"2024-09-11T01:17:15.569244+0000","last_clean":"2024-09-11T01:17:15.569244+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:15.569244+0000","last_undegraded":"2024-09-11T01:17:15.569244+0000","last_fullsized":"2024-09-11T01:17:15.569244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7876,"log_dups_size":0,"ondisk_log_size":7876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231907,"num_objects":7870,"num_object_clones":0,"num_object_copies":15740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4674,"num_write_kb":18696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7924","reported_seq":8049,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.538013+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:15.538013+0000","last_peered":"2024-09-11T01:17:15.538013+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:15.538013+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436707,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4412,"num_write_kb":17648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8126","reported_seq":8250,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.365592+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:15.365592+0000","last_peered":"2024-09-11T01:17:15.365592+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:15.365592+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8126,"log_dups_size":0,"ondisk_log_size":8126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33260486,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3839},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7975","reported_seq":8085,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.544695+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:15.544695+0000","last_peered":"2024-09-11T01:17:15.544695+0000","last_clean":"2024-09-11T01:17:15.544695+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:15.544695+0000","last_undegraded":"2024-09-11T01:17:15.544695+0000","last_fullsized":"2024-09-11T01:17:15.544695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4928,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8098","reported_seq":15010,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:20.091467+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:20.091467+0000","last_peered":"2024-09-11T01:17:20.091467+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:20.091467+0000","last_undegraded":"2024-09-11T01:17:20.091467+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5965,"num_write_kb":23860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3419,"num_bytes_recovered":13996515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":706},{"shards":"2","objects":7386}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8046","reported_seq":8164,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.558766+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:15.558766+0000","last_peered":"2024-09-11T01:17:15.558766+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:15.558766+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7930","reported_seq":8015,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.071696+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:20.071696+0000","last_peered":"2024-09-11T01:17:20.071696+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:20.071696+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7930,"log_dups_size":0,"ondisk_log_size":7930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32461283,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5699,"num_write_kb":22796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3899}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8186","reported_seq":8270,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.043638+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:20.043638+0000","last_peered":"2024-09-11T01:17:20.043638+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:20.043638+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8186,"log_dups_size":0,"ondisk_log_size":8186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4003}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7954","reported_seq":8075,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.511983+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:15.511983+0000","last_peered":"2024-09-11T01:17:15.511983+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:15.511983+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7954,"log_dups_size":0,"ondisk_log_size":7954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8121","reported_seq":8246,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.579196+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:15.579196+0000","last_peered":"2024-09-11T01:17:15.579196+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:15.579196+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33230848,"num_objects":8113,"num_object_clones":0,"num_object_copies":16226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7999","reported_seq":8123,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.592238+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:15.592238+0000","last_peered":"2024-09-11T01:17:15.592238+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:15.592238+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7999,"log_dups_size":0,"ondisk_log_size":7999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32747520,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4534,"num_write_kb":18136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7888","reported_seq":8014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.477681+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:15.477681+0000","last_peered":"2024-09-11T01:17:15.477681+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:15.477681+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7888,"log_dups_size":0,"ondisk_log_size":7888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32261062,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3724},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":524661470,"num_objects":128100,"num_object_clones":0,"num_object_copies":256200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":128100,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":87235,"num_write_kb":348940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3420,"num_bytes_recovered":14000611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064366080,"data_stored":1064214334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128172,"ondisk_log_size":128172,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617012,"kb_used_data":390948,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93754828,"statfs":{"total":96636764160,"available":96004943872,"internally_reserved":0,"allocated":400330752,"data_stored":400073848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789977,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376200,"kb_used_data":200120,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93995640,"statfs":{"total":96636764160,"available":96251535360,"internally_reserved":0,"allocated":204922880,"data_stored":204618543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709372,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93662468,"statfs":{"total":96636764160,"available":95910367232,"internally_reserved":0,"allocated":458010624,"data_stored":457749643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458698752,"data_stored":458662622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205262848,"data_stored":205183362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400404480,"data_stored":400368350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:21.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:22.302 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:22.304 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:22.327 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-11T01:17:20.462313+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":524661489,"num_objects":128102,"num_object_clones":0,"num_object_copies":256204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":128102,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":87237,"num_write_kb":348942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":14000630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128175,"ondisk_log_size":128175,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1702584,"kb_used_data":1038344,"kb_used_omap":12,"kb_used_meta":664179,"kb_avail":281412936,"statfs":{"total":289910292480,"available":288166846464,"internally_reserved":0,"allocated":1063264256,"data_stored":1062442034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":680119303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,11,6,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-540,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5811,"num_write_kb":23244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":540,"num_bytes_recovered":2211840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006751"},"pg_stats":[{"pgid":"2.f","version":"39'7890","reported_seq":7954,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.574186+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:15.574186+0000","last_peered":"2024-09-11T01:17:15.574186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:15.574186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":31560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8116","reported_seq":8187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.044266+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:20.044266+0000","last_peered":"2024-09-11T01:17:20.044266+0000","last_clean":"2024-09-11T01:17:20.044266+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:20.044266+0000","last_undegraded":"2024-09-11T01:17:20.044266+0000","last_fullsized":"2024-09-11T01:17:20.044266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33219526,"num_objects":8112,"num_object_clones":0,"num_object_copies":16224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7989","reported_seq":8063,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.039005+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:20.039005+0000","last_peered":"2024-09-11T01:17:20.039005+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:20.039005+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7989,"log_dups_size":0,"ondisk_log_size":7989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32706560,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6050,"num_write_kb":24200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3980}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8054","reported_seq":8087,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.086460+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:20.086460+0000","last_peered":"2024-09-11T01:17:20.086460+0000","last_clean":"2024-09-11T01:17:20.086460+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:20.086460+0000","last_undegraded":"2024-09-11T01:17:20.086460+0000","last_fullsized":"2024-09-11T01:17:20.086460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8054,"log_dups_size":0,"ondisk_log_size":8054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989184,"num_objects":8054,"num_object_clones":0,"num_object_copies":16108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8054,"num_write_kb":32216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'7876","reported_seq":7988,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.569244+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:15.569244+0000","last_peered":"2024-09-11T01:17:15.569244+0000","last_clean":"2024-09-11T01:17:15.569244+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:15.569244+0000","last_undegraded":"2024-09-11T01:17:15.569244+0000","last_fullsized":"2024-09-11T01:17:15.569244+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":7876,"log_dups_size":0,"ondisk_log_size":7876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231907,"num_objects":7870,"num_object_clones":0,"num_object_copies":15740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4674,"num_write_kb":18696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'7924","reported_seq":8049,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.538013+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:15.538013+0000","last_peered":"2024-09-11T01:17:15.538013+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:15.538013+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":7924,"log_dups_size":0,"ondisk_log_size":7924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436707,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4412,"num_write_kb":17648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3805},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8126","reported_seq":8250,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.365592+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:15.365592+0000","last_peered":"2024-09-11T01:17:15.365592+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:15.365592+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8126,"log_dups_size":0,"ondisk_log_size":8126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33260486,"num_objects":8122,"num_object_clones":0,"num_object_copies":16244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8122,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4879,"num_write_kb":19516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3839},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'7975","reported_seq":8085,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.544695+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:15.544695+0000","last_peered":"2024-09-11T01:17:15.544695+0000","last_clean":"2024-09-11T01:17:15.544695+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:15.544695+0000","last_undegraded":"2024-09-11T01:17:15.544695+0000","last_fullsized":"2024-09-11T01:17:15.544695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":7975,"log_dups_size":0,"ondisk_log_size":7975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4928,"num_write_kb":19712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8098","reported_seq":15010,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:20.091467+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:20.091467+0000","last_peered":"2024-09-11T01:17:20.091467+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:20.091467+0000","last_undegraded":"2024-09-11T01:17:20.091467+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5965,"num_write_kb":23860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3419,"num_bytes_recovered":13996515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":706},{"shards":"2","objects":7386}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8046","reported_seq":8164,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.558766+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:15.558766+0000","last_peered":"2024-09-11T01:17:15.558766+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:15.558766+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8046,"log_dups_size":0,"ondisk_log_size":8046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4410,"num_write_kb":17640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3933},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7930","reported_seq":8015,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.071696+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:20.071696+0000","last_peered":"2024-09-11T01:17:20.071696+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:20.071696+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7930,"log_dups_size":0,"ondisk_log_size":7930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32461283,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5699,"num_write_kb":22796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3899}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8186","reported_seq":8270,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.043638+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:20.043638+0000","last_peered":"2024-09-11T01:17:20.043638+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:20.043638+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8186,"log_dups_size":0,"ondisk_log_size":8186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4003}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'7954","reported_seq":8075,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.511983+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:15.511983+0000","last_peered":"2024-09-11T01:17:15.511983+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:15.511983+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":7954,"log_dups_size":0,"ondisk_log_size":7954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32571392,"num_objects":7952,"num_object_clones":0,"num_object_copies":15904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":18576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8121","reported_seq":8246,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.579196+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:15.579196+0000","last_peered":"2024-09-11T01:17:15.579196+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:15.579196+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8121,"log_dups_size":0,"ondisk_log_size":8121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33230848,"num_objects":8113,"num_object_clones":0,"num_object_copies":16226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3921},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'7999","reported_seq":8123,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.592238+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:15.592238+0000","last_peered":"2024-09-11T01:17:15.592238+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:15.592238+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":7999,"log_dups_size":0,"ondisk_log_size":7999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32747520,"num_objects":7995,"num_object_clones":0,"num_object_copies":15990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4534,"num_write_kb":18136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'7888","reported_seq":8014,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.477681+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:15.477681+0000","last_peered":"2024-09-11T01:17:15.477681+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:15.477681+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":7888,"log_dups_size":0,"ondisk_log_size":7888,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32261062,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3724},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":524661470,"num_objects":128100,"num_object_clones":0,"num_object_copies":256200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":128100,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":87235,"num_write_kb":348940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3420,"num_bytes_recovered":14000611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064366080,"data_stored":1064214334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128172,"ondisk_log_size":128172,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617012,"kb_used_data":390948,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93754828,"statfs":{"total":96636764160,"available":96004943872,"internally_reserved":0,"allocated":400330752,"data_stored":400073848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789977,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376200,"kb_used_data":200120,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93995640,"statfs":{"total":96636764160,"available":96251535360,"internally_reserved":0,"allocated":204922880,"data_stored":204618543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709372,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93662468,"statfs":{"total":96636764160,"available":95910367232,"internally_reserved":0,"allocated":458010624,"data_stored":457749643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":458698752,"data_stored":458662622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205262848,"data_stored":205183362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400404480,"data_stored":400368350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'7890", 'reported_seq': 7954, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.574186+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:15.574186+0000', 'last_peered': '2024-09-11T01:17:15.574186+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:15.574186+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 7890, 'log_dups_size': 0, 'ondisk_log_size': 7890, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32317440, 'num_objects': 7890, 'num_object_clones': 0, 'num_object_copies': 15780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7890, 'num_write_kb': 31560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3767}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'7989", 'reported_seq': 8063, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.039005+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:20.039005+0000', 'last_peered': '2024-09-11T01:17:20.039005+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:20.039005+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 7989, 'log_dups_size': 0, 'ondisk_log_size': 7989, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32706560, 'num_objects': 7985, 'num_object_clones': 0, 'num_object_copies': 15970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7985, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6050, 'num_write_kb': 24200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 3980}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'7924", 'reported_seq': 8049, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.538013+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:15.538013+0000', 'last_peered': '2024-09-11T01:17:15.538013+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:15.538013+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 7924, 'log_dups_size': 0, 'ondisk_log_size': 7924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32436707, 'num_objects': 7920, 'num_object_clones': 0, 'num_object_copies': 15840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7920, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4412, 'num_write_kb': 17648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3805}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'8126", 'reported_seq': 8250, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.365592+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:15.365592+0000', 'last_peered': '2024-09-11T01:17:15.365592+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:15.365592+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 8126, 'log_dups_size': 0, 'ondisk_log_size': 8126, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33260486, 'num_objects': 8122, 'num_object_clones': 0, 'num_object_copies': 16244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8122, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 4879, 'num_write_kb': 19516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3839}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.329 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'8098", 'reported_seq': 15010, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:20.091467+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:17:20.091467+0000', 'last_peered': '2024-09-11T01:17:20.091467+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:17:20.091467+0000', 'last_undegraded': '2024-09-11T01:17:20.091467+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 8098, 'log_dups_size': 0, 'ondisk_log_size': 8098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33141219, 'num_objects': 8092, 'num_object_clones': 0, 'num_object_copies': 16184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 706, 'num_objects_unfound': 0, 'num_objects_dirty': 8092, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5965, 'num_write_kb': 23860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3419, 'num_bytes_recovered': 13996515, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 706}, {'shards': '2', 'objects': 7386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'8046", 'reported_seq': 8164, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.558766+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:15.558766+0000', 'last_peered': '2024-09-11T01:17:15.558766+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:15.558766+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8046, 'log_dups_size': 0, 'ondisk_log_size': 8046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32923648, 'num_objects': 8038, 'num_object_clones': 0, 'num_object_copies': 16076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4410, 'num_write_kb': 17640, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3933}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'7930", 'reported_seq': 8015, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.071696+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:20.071696+0000', 'last_peered': '2024-09-11T01:17:20.071696+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:20.071696+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 7930, 'log_dups_size': 0, 'ondisk_log_size': 7930, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32461283, 'num_objects': 7926, 'num_object_clones': 0, 'num_object_copies': 15852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7926, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5699, 'num_write_kb': 22796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 3899}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'8186", 'reported_seq': 8270, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.043638+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:20.043638+0000', 'last_peered': '2024-09-11T01:17:20.043638+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:20.043638+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8186, 'log_dups_size': 0, 'ondisk_log_size': 8186, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33505280, 'num_objects': 8180, 'num_object_clones': 0, 'num_object_copies': 16360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5823, 'num_write_kb': 23292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4003}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'7954", 'reported_seq': 8075, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.511983+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:15.511983+0000', 'last_peered': '2024-09-11T01:17:15.511983+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:15.511983+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 7954, 'log_dups_size': 0, 'ondisk_log_size': 7954, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32571392, 'num_objects': 7952, 'num_object_clones': 0, 'num_object_copies': 15904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4644, 'num_write_kb': 18576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3788}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'8121", 'reported_seq': 8246, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.579196+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:15.579196+0000', 'last_peered': '2024-09-11T01:17:15.579196+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:15.579196+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 8121, 'log_dups_size': 0, 'ondisk_log_size': 8121, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33230848, 'num_objects': 8113, 'num_object_clones': 0, 'num_object_copies': 16226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4514, 'num_write_kb': 18056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3921}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.330 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:22.331 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'7999", 'reported_seq': 8123, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.592238+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:15.592238+0000', 'last_peered': '2024-09-11T01:17:15.592238+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:15.592238+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 7999, 'log_dups_size': 0, 'ondisk_log_size': 7999, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32747520, 'num_objects': 7995, 'num_object_clones': 0, 'num_object_copies': 15990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7995, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4534, 'num_write_kb': 18136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.331 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:22.331 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'7888", 'reported_seq': 8014, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.477681+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:15.477681+0000', 'last_peered': '2024-09-11T01:17:15.477681+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:15.477681+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 7888, 'log_dups_size': 0, 'ondisk_log_size': 7888, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32261062, 'num_objects': 7878, 'num_object_clones': 0, 'num_object_copies': 15756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7878, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4545, 'num_write_kb': 18180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3724}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:22.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:22.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 166 20 130459 130439 3.06878 1.76562 0.017917 0.0254523 2024-09-11T01:17:23.053 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:23.054 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/259158 objects degraded (17.560%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":243,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129579,"data_bytes":530711281,"bytes_used":1773510656,"bytes_avail":288136781824,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":259158,"degraded_ratio":0.17559944126748936,"misplaced_objects":706,"misplaced_total":259158,"misplaced_ratio":0.0027242068545057455,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":123151,"recovering_keys_per_sec":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"write_bytes_sec":2176816,"read_op_per_sec":0,"write_op_per_sec":531},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:23.077 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:23.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:23.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 167 20 130897 130877 3.06065 1.71094 0.13756 0.0255168 2024-09-11T01:17:23.717 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:23.717 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:23.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-09-11T01:17:22.462821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530711281,"num_objects":129579,"num_object_clones":0,"num_object_copies":259158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":129579,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":88714,"num_write_kb":354850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":14000630,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129652,"ondisk_log_size":129652,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1731944,"kb_used_data":1045496,"kb_used_omap":12,"kb_used_meta":686387,"kb_avail":281383576,"statfs":{"total":289910292480,"available":288136781824,"internally_reserved":0,"allocated":1070587904,"data_stored":1069765682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":702860295},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,11,2,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-361,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6381,"num_write_kb":25524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006787"},"pg_stats":[{"pgid":"2.f","version":"39'8025","reported_seq":8088,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.595014+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:20.595014+0000","last_peered":"2024-09-11T01:17:20.595014+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:20.595014+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8025,"log_dups_size":0,"ondisk_log_size":8025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32870400,"num_objects":8025,"num_object_clones":0,"num_object_copies":16050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8025,"num_write_kb":32100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3902},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8116","reported_seq":8187,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.044266+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:20.044266+0000","last_peered":"2024-09-11T01:17:20.044266+0000","last_clean":"2024-09-11T01:17:20.044266+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:20.044266+0000","last_undegraded":"2024-09-11T01:17:20.044266+0000","last_fullsized":"2024-09-11T01:17:20.044266+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8116,"log_dups_size":0,"ondisk_log_size":8116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33219526,"num_objects":8112,"num_object_clones":0,"num_object_copies":16224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8112,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'7989","reported_seq":8063,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.039005+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:20.039005+0000","last_peered":"2024-09-11T01:17:20.039005+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:20.039005+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":7989,"log_dups_size":0,"ondisk_log_size":7989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32706560,"num_objects":7985,"num_object_clones":0,"num_object_copies":15970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6050,"num_write_kb":24200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":3980}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8054","reported_seq":8087,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.086460+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:20.086460+0000","last_peered":"2024-09-11T01:17:20.086460+0000","last_clean":"2024-09-11T01:17:20.086460+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:20.086460+0000","last_undegraded":"2024-09-11T01:17:20.086460+0000","last_fullsized":"2024-09-11T01:17:20.086460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8054,"log_dups_size":0,"ondisk_log_size":8054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32989184,"num_objects":8054,"num_object_clones":0,"num_object_copies":16108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8054,"num_write_kb":32216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8047","reported_seq":8159,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.523331+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:20.523331+0000","last_peered":"2024-09-11T01:17:20.523331+0000","last_clean":"2024-09-11T01:17:20.523331+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:20.523331+0000","last_undegraded":"2024-09-11T01:17:20.523331+0000","last_fullsized":"2024-09-11T01:17:20.523331+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32932323,"num_objects":8041,"num_object_clones":0,"num_object_copies":16082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4845,"num_write_kb":19380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8098","reported_seq":8223,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.606526+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:20.606526+0000","last_peered":"2024-09-11T01:17:20.606526+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:20.606526+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33149411,"num_objects":8094,"num_object_clones":0,"num_object_copies":16188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4586,"num_write_kb":18344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8273","reported_seq":8396,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.629514+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:20.629514+0000","last_peered":"2024-09-11T01:17:20.629514+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:20.629514+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8273,"log_dups_size":0,"ondisk_log_size":8273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33862598,"num_objects":8269,"num_object_clones":0,"num_object_copies":16538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8269,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5026,"num_write_kb":20104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3986},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8107","reported_seq":8217,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.574781+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:20.574781+0000","last_peered":"2024-09-11T01:17:20.574781+0000","last_clean":"2024-09-11T01:17:20.574781+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:20.574781+0000","last_undegraded":"2024-09-11T01:17:20.574781+0000","last_fullsized":"2024-09-11T01:17:20.574781+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8107,"log_dups_size":0,"ondisk_log_size":8107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33198080,"num_objects":8105,"num_object_clones":0,"num_object_copies":16210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5060,"num_write_kb":20240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8098","reported_seq":15010,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:20.091467+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:20.091467+0000","last_peered":"2024-09-11T01:17:20.091467+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:20.091467+0000","last_undegraded":"2024-09-11T01:17:20.091467+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8098,"log_dups_size":0,"ondisk_log_size":8098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5965,"num_write_kb":23860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3419,"num_bytes_recovered":13996515,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":706},{"shards":"2","objects":7386}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8205","reported_seq":8325,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.603761+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:20.603761+0000","last_peered":"2024-09-11T01:17:20.603761+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:20.603761+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8205,"log_dups_size":0,"ondisk_log_size":8205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33574912,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4569,"num_write_kb":18276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4092},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'7930","reported_seq":8015,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.071696+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:20.071696+0000","last_peered":"2024-09-11T01:17:20.071696+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:20.071696+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":7930,"log_dups_size":0,"ondisk_log_size":7930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32461283,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5699,"num_write_kb":22796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":3899}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.047235+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.047235+0000","last_peered":"2024-09-11T01:15:20.047235+0000","last_clean":"2024-09-11T01:15:20.047235+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.047235+0000","last_undegraded":"2024-09-11T01:15:20.047235+0000","last_fullsized":"2024-09-11T01:15:20.047235+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8186","reported_seq":8270,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.043638+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:20.043638+0000","last_peered":"2024-09-11T01:17:20.043638+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:20.043638+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8186,"log_dups_size":0,"ondisk_log_size":8186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505280,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4003}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8089","reported_seq":8210,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.578901+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:20.578901+0000","last_peered":"2024-09-11T01:17:20.578901+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:20.578901+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8089,"log_dups_size":0,"ondisk_log_size":8089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33124352,"num_objects":8087,"num_object_clones":0,"num_object_copies":16174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4779,"num_write_kb":19116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3923},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.197892+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.197892+0000","last_peered":"2024-09-11T01:15:20.197892+0000","last_clean":"2024-09-11T01:15:20.197892+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.197892+0000","last_undegraded":"2024-09-11T01:15:20.197892+0000","last_fullsized":"2024-09-11T01:15:20.197892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8262","reported_seq":8387,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.578526+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:20.578526+0000","last_peered":"2024-09-11T01:17:20.578526+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:20.578526+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8262,"log_dups_size":0,"ondisk_log_size":8262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33808384,"num_objects":8254,"num_object_clones":0,"num_object_copies":16508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4655,"num_write_kb":18620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4062},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8140","reported_seq":8264,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.522610+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:20.522610+0000","last_peered":"2024-09-11T01:17:20.522610+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:20.522610+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8140,"log_dups_size":0,"ondisk_log_size":8140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4675,"num_write_kb":18700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3988},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.198717+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.198717+0000","last_peered":"2024-09-11T01:15:20.198717+0000","last_clean":"2024-09-11T01:15:20.198717+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.198717+0000","last_undegraded":"2024-09-11T01:15:20.198717+0000","last_fullsized":"2024-09-11T01:15:20.198717+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8030","reported_seq":8156,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.615225+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:20.615225+0000","last_peered":"2024-09-11T01:17:20.615225+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:20.615225+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8030,"log_dups_size":0,"ondisk_log_size":8030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32842694,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4687,"num_write_kb":18748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3866},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530711262,"num_objects":129577,"num_object_clones":0,"num_object_copies":259154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":706,"num_objects_unfound":0,"num_objects_dirty":129577,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":88712,"num_write_kb":354848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3420,"num_bytes_recovered":14000611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1071616000,"data_stored":1071464254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129649,"ondisk_log_size":129649,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":617012,"kb_used_data":390948,"kb_used_omap":4,"kb_used_meta":226043,"kb_avail":93754828,"statfs":{"total":96636764160,"available":96004943872,"internally_reserved":0,"allocated":400330752,"data_stored":400073848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":231468718},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789977,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376200,"kb_used_data":200120,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93995640,"statfs":{"total":96636764160,"available":96251535360,"internally_reserved":0,"allocated":204922880,"data_stored":204618543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738732,"kb_used_data":454428,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93633108,"statfs":{"total":96636764160,"available":95880302592,"internally_reserved":0,"allocated":465334272,"data_stored":465073291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,0,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":465948672,"data_stored":465912542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":205262848,"data_stored":205183362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400404480,"data_stored":400368350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:24.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 168 20 131365 131345 3.05332 1.82812 0.0189215 0.0255777 2024-09-11T01:17:25.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 169 20 131866 131846 3.04683 1.95703 0.0412572 0.025634 2024-09-11T01:17:26.347 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 170 20 132370 132350 3.04049 1.96875 0.0288671 0.0256882 2024-09-11T01:17:26.742 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:27.315 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:27.316 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:27.344 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-11T01:17:26.463863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540406513,"num_objects":131946,"num_object_clones":0,"num_object_copies":263892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":131946,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":91081,"num_write_kb":364318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3629,"num_bytes_recovered":14852598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132019,"ondisk_log_size":132019,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786404,"kb_used_data":1065264,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281329116,"statfs":{"total":289910292480,"available":288081014784,"internally_reserved":0,"allocated":1090830336,"data_stored":1090008114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":738380807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,2,8,17,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29413376,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-569,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":28724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":569,"num_bytes_recovered":2330624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006818"},"pg_stats":[{"pgid":"2.f","version":"39'8154","reported_seq":8218,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.648032+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:25.648032+0000","last_peered":"2024-09-11T01:17:25.648032+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:25.648032+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8154,"log_dups_size":0,"ondisk_log_size":8154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8154,"num_write_kb":32616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4031},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8252","reported_seq":8321,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.093052+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:25.093052+0000","last_peered":"2024-09-11T01:17:25.093052+0000","last_clean":"2024-09-11T01:17:25.093052+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:25.093052+0000","last_undegraded":"2024-09-11T01:17:25.093052+0000","last_fullsized":"2024-09-11T01:17:25.093052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6350,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8130","reported_seq":8203,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.104597+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:25.104597+0000","last_peered":"2024-09-11T01:17:25.104597+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:25.104597+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8130,"log_dups_size":0,"ondisk_log_size":8130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284096,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6191,"num_write_kb":24764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8192","reported_seq":8228,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.983506+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:24.983506+0000","last_peered":"2024-09-11T01:17:24.983506+0000","last_clean":"2024-09-11T01:17:24.983506+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:24.983506+0000","last_undegraded":"2024-09-11T01:17:24.983506+0000","last_fullsized":"2024-09-11T01:17:24.983506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8192,"log_dups_size":0,"ondisk_log_size":8192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8192,"num_object_clones":0,"num_object_copies":16384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8196","reported_seq":8308,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.665627+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:25.665627+0000","last_peered":"2024-09-11T01:17:25.665627+0000","last_clean":"2024-09-11T01:17:25.665627+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:25.665627+0000","last_undegraded":"2024-09-11T01:17:25.665627+0000","last_fullsized":"2024-09-11T01:17:25.665627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8196,"log_dups_size":0,"ondisk_log_size":8196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33542627,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4994,"num_write_kb":19976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8224","reported_seq":8349,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.555514+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:25.555514+0000","last_peered":"2024-09-11T01:17:25.555514+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:25.555514+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8224,"log_dups_size":0,"ondisk_log_size":8224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665507,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4712,"num_write_kb":18848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4105},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8434","reported_seq":8558,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.638568+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:25.638568+0000","last_peered":"2024-09-11T01:17:25.638568+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:25.638568+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8434,"log_dups_size":0,"ondisk_log_size":8434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34522054,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4147},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8251","reported_seq":8356,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.611578+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:25.611578+0000","last_peered":"2024-09-11T01:17:25.611578+0000","last_clean":"2024-09-11T01:17:25.611578+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:25.611578+0000","last_undegraded":"2024-09-11T01:17:25.611578+0000","last_fullsized":"2024-09-11T01:17:25.611578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8251,"log_dups_size":0,"ondisk_log_size":8251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5204,"num_write_kb":20816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8243","reported_seq":15571,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:25.098916+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:25.098916+0000","last_peered":"2024-09-11T01:17:25.098916+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:25.098916+0000","last_undegraded":"2024-09-11T01:17:25.098916+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8243,"log_dups_size":0,"ondisk_log_size":8243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33735139,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6110,"num_write_kb":24440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3627,"num_bytes_recovered":14848483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":7739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8344","reported_seq":8464,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.618530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:25.618530+0000","last_peered":"2024-09-11T01:17:25.618530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:25.618530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8344,"log_dups_size":0,"ondisk_log_size":8344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8078","reported_seq":8163,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.064007+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:25.064007+0000","last_peered":"2024-09-11T01:17:25.064007+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:25.064007+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8078,"log_dups_size":0,"ondisk_log_size":8078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067491,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.408712+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.408712+0000","last_peered":"2024-09-11T01:15:20.408712+0000","last_clean":"2024-09-11T01:15:20.408712+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.408712+0000","last_undegraded":"2024-09-11T01:15:20.408712+0000","last_fullsized":"2024-09-11T01:15:20.408712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8346","reported_seq":8430,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.047253+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:25.047253+0000","last_peered":"2024-09-11T01:17:25.047253+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:25.047253+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5983,"num_write_kb":23932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8269","reported_seq":8390,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.694398+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:25.694398+0000","last_peered":"2024-09-11T01:17:25.694398+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:25.694398+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8269,"log_dups_size":0,"ondisk_log_size":8269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4959,"num_write_kb":19836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4103},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.481918+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.481918+0000","last_peered":"2024-09-11T01:15:20.481918+0000","last_clean":"2024-09-11T01:15:20.481918+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.481918+0000","last_undegraded":"2024-09-11T01:15:20.481918+0000","last_fullsized":"2024-09-11T01:15:20.481918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8432","reported_seq":8557,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.664357+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:25.664357+0000","last_peered":"2024-09-11T01:17:25.664357+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:25.664357+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8432,"log_dups_size":0,"ondisk_log_size":8432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34504704,"num_objects":8424,"num_object_clones":0,"num_object_copies":16848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4825,"num_write_kb":19300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4232},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8285","reported_seq":8409,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.582213+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:25.582213+0000","last_peered":"2024-09-11T01:17:25.582213+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:25.582213+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8285,"log_dups_size":0,"ondisk_log_size":8285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4820,"num_write_kb":19280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.507048+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.507048+0000","last_peered":"2024-09-11T01:15:20.507048+0000","last_clean":"2024-09-11T01:15:20.507048+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.507048+0000","last_undegraded":"2024-09-11T01:15:20.507048+0000","last_fullsized":"2024-09-11T01:15:20.507048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8186","reported_seq":8312,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.661792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:25.661792+0000","last_peered":"2024-09-11T01:17:25.661792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:25.661792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8186,"log_dups_size":0,"ondisk_log_size":8186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33481670,"num_objects":8176,"num_object_clones":0,"num_object_copies":16352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4843,"num_write_kb":19372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4022},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":540406494,"num_objects":131944,"num_object_clones":0,"num_object_copies":263888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":131944,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":91079,"num_write_kb":364316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3628,"num_bytes_recovered":14852579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1091891200,"data_stored":1091739454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132016,"ondisk_log_size":132016,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625692,"kb_used_data":395464,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93746148,"statfs":{"total":96636764160,"available":95996055552,"internally_reserved":0,"allocated":404955136,"data_stored":404698232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406520,"kb_used_data":208232,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93965320,"statfs":{"total":96636764160,"available":96220487680,"internally_reserved":0,"allocated":213229568,"data_stored":212925231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754192,"kb_used_data":461568,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93617648,"statfs":{"total":96636764160,"available":95864471552,"internally_reserved":0,"allocated":472645632,"data_stored":472384651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473247744,"data_stored":473211614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213565440,"data_stored":213485954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405078016,"data_stored":405041886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:27.345 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:27.349 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 171 20 132823 132803 3.03302 1.76953 0.101736 0.0257475 2024-09-11T01:17:27.953 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:27.955 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:27.982 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-11T01:17:26.463863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540406513,"num_objects":131946,"num_object_clones":0,"num_object_copies":263892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":131946,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":91081,"num_write_kb":364318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3629,"num_bytes_recovered":14852598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132019,"ondisk_log_size":132019,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786404,"kb_used_data":1065264,"kb_used_omap":12,"kb_used_meta":721075,"kb_avail":281329116,"statfs":{"total":289910292480,"available":288081014784,"internally_reserved":0,"allocated":1090830336,"data_stored":1090008114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":738380807},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,2,8,17,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29413376,"num_objects":7181,"num_object_clones":0,"num_object_copies":14362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-569,"num_objects_unfound":0,"num_objects_dirty":7181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":28724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":569,"num_bytes_recovered":2330624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006818"},"pg_stats":[{"pgid":"2.f","version":"39'8154","reported_seq":8218,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.648032+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:25.648032+0000","last_peered":"2024-09-11T01:17:25.648032+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:25.648032+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8154,"log_dups_size":0,"ondisk_log_size":8154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8154,"num_write_kb":32616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4031},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8252","reported_seq":8321,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.093052+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:25.093052+0000","last_peered":"2024-09-11T01:17:25.093052+0000","last_clean":"2024-09-11T01:17:25.093052+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:25.093052+0000","last_undegraded":"2024-09-11T01:17:25.093052+0000","last_fullsized":"2024-09-11T01:17:25.093052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6350,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8130","reported_seq":8203,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.104597+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:25.104597+0000","last_peered":"2024-09-11T01:17:25.104597+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:25.104597+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8130,"log_dups_size":0,"ondisk_log_size":8130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284096,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6191,"num_write_kb":24764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8192","reported_seq":8228,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.983506+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:24.983506+0000","last_peered":"2024-09-11T01:17:24.983506+0000","last_clean":"2024-09-11T01:17:24.983506+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:24.983506+0000","last_undegraded":"2024-09-11T01:17:24.983506+0000","last_fullsized":"2024-09-11T01:17:24.983506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8192,"log_dups_size":0,"ondisk_log_size":8192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8192,"num_object_clones":0,"num_object_copies":16384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8196","reported_seq":8308,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.665627+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:25.665627+0000","last_peered":"2024-09-11T01:17:25.665627+0000","last_clean":"2024-09-11T01:17:25.665627+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:25.665627+0000","last_undegraded":"2024-09-11T01:17:25.665627+0000","last_fullsized":"2024-09-11T01:17:25.665627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8196,"log_dups_size":0,"ondisk_log_size":8196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33542627,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4994,"num_write_kb":19976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8224","reported_seq":8349,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.555514+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:25.555514+0000","last_peered":"2024-09-11T01:17:25.555514+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:25.555514+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8224,"log_dups_size":0,"ondisk_log_size":8224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665507,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4712,"num_write_kb":18848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4105},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8434","reported_seq":8558,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.638568+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:25.638568+0000","last_peered":"2024-09-11T01:17:25.638568+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:25.638568+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8434,"log_dups_size":0,"ondisk_log_size":8434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34522054,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4147},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8251","reported_seq":8356,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.611578+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:25.611578+0000","last_peered":"2024-09-11T01:17:25.611578+0000","last_clean":"2024-09-11T01:17:25.611578+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:25.611578+0000","last_undegraded":"2024-09-11T01:17:25.611578+0000","last_fullsized":"2024-09-11T01:17:25.611578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8251,"log_dups_size":0,"ondisk_log_size":8251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5204,"num_write_kb":20816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8243","reported_seq":15571,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:25.098916+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:25.098916+0000","last_peered":"2024-09-11T01:17:25.098916+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:25.098916+0000","last_undegraded":"2024-09-11T01:17:25.098916+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8243,"log_dups_size":0,"ondisk_log_size":8243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33735139,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6110,"num_write_kb":24440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3627,"num_bytes_recovered":14848483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":7739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8344","reported_seq":8464,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.618530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:25.618530+0000","last_peered":"2024-09-11T01:17:25.618530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:25.618530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8344,"log_dups_size":0,"ondisk_log_size":8344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8078","reported_seq":8163,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.064007+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:25.064007+0000","last_peered":"2024-09-11T01:17:25.064007+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:25.064007+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8078,"log_dups_size":0,"ondisk_log_size":8078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067491,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.408712+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.408712+0000","last_peered":"2024-09-11T01:15:20.408712+0000","last_clean":"2024-09-11T01:15:20.408712+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.408712+0000","last_undegraded":"2024-09-11T01:15:20.408712+0000","last_fullsized":"2024-09-11T01:15:20.408712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8346","reported_seq":8430,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.047253+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:25.047253+0000","last_peered":"2024-09-11T01:17:25.047253+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:25.047253+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5983,"num_write_kb":23932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8269","reported_seq":8390,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.694398+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:25.694398+0000","last_peered":"2024-09-11T01:17:25.694398+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:25.694398+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8269,"log_dups_size":0,"ondisk_log_size":8269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4959,"num_write_kb":19836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4103},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.481918+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.481918+0000","last_peered":"2024-09-11T01:15:20.481918+0000","last_clean":"2024-09-11T01:15:20.481918+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.481918+0000","last_undegraded":"2024-09-11T01:15:20.481918+0000","last_fullsized":"2024-09-11T01:15:20.481918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8432","reported_seq":8557,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.664357+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:25.664357+0000","last_peered":"2024-09-11T01:17:25.664357+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:25.664357+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8432,"log_dups_size":0,"ondisk_log_size":8432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34504704,"num_objects":8424,"num_object_clones":0,"num_object_copies":16848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4825,"num_write_kb":19300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4232},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8285","reported_seq":8409,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.582213+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:25.582213+0000","last_peered":"2024-09-11T01:17:25.582213+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:25.582213+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8285,"log_dups_size":0,"ondisk_log_size":8285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4820,"num_write_kb":19280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.507048+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.507048+0000","last_peered":"2024-09-11T01:15:20.507048+0000","last_clean":"2024-09-11T01:15:20.507048+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.507048+0000","last_undegraded":"2024-09-11T01:15:20.507048+0000","last_fullsized":"2024-09-11T01:15:20.507048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.996121+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.996121+0000","last_peered":"2024-09-11T01:15:19.996121+0000","last_clean":"2024-09-11T01:15:19.996121+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.996121+0000","last_undegraded":"2024-09-11T01:15:19.996121+0000","last_fullsized":"2024-09-11T01:15:19.996121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8186","reported_seq":8312,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.661792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:25.661792+0000","last_peered":"2024-09-11T01:17:25.661792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:25.661792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8186,"log_dups_size":0,"ondisk_log_size":8186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33481670,"num_objects":8176,"num_object_clones":0,"num_object_copies":16352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4843,"num_write_kb":19372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4022},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":540406494,"num_objects":131944,"num_object_clones":0,"num_object_copies":263888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":131944,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":91079,"num_write_kb":364316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3628,"num_bytes_recovered":14852579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1091891200,"data_stored":1091739454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132016,"ondisk_log_size":132016,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625692,"kb_used_data":395464,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93746148,"statfs":{"total":96636764160,"available":95996055552,"internally_reserved":0,"allocated":404955136,"data_stored":404698232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789978,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406520,"kb_used_data":208232,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93965320,"statfs":{"total":96636764160,"available":96220487680,"internally_reserved":0,"allocated":213229568,"data_stored":212925231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754192,"kb_used_data":461568,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93617648,"statfs":{"total":96636764160,"available":95864471552,"internally_reserved":0,"allocated":472645632,"data_stored":472384651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473247744,"data_stored":473211614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":213565440,"data_stored":213485954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405078016,"data_stored":405041886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:27.984 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:27.985 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'8154", 'reported_seq': 8218, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.648032+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:25.648032+0000', 'last_peered': '2024-09-11T01:17:25.648032+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:25.648032+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 8154, 'log_dups_size': 0, 'ondisk_log_size': 8154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33398784, 'num_objects': 8154, 'num_object_clones': 0, 'num_object_copies': 16308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8154, 'num_write_kb': 32616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4031}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.985 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:27.985 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'8130", 'reported_seq': 8203, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.104597+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:25.104597+0000', 'last_peered': '2024-09-11T01:17:25.104597+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:25.104597+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 8130, 'log_dups_size': 0, 'ondisk_log_size': 8130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33284096, 'num_objects': 8126, 'num_object_clones': 0, 'num_object_copies': 16252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6191, 'num_write_kb': 24764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 4121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:27.985 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:27.986 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'8224", 'reported_seq': 8349, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.555514+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:25.555514+0000', 'last_peered': '2024-09-11T01:17:25.555514+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:25.555514+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 8224, 'log_dups_size': 0, 'ondisk_log_size': 8224, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33665507, 'num_objects': 8220, 'num_object_clones': 0, 'num_object_copies': 16440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8220, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4712, 'num_write_kb': 18848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4105}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.986 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:27.986 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'8434", 'reported_seq': 8558, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.638568+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:25.638568+0000', 'last_peered': '2024-09-11T01:17:25.638568+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:25.638568+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 8434, 'log_dups_size': 0, 'ondisk_log_size': 8434, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34522054, 'num_objects': 8430, 'num_object_clones': 0, 'num_object_copies': 16860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8430, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5187, 'num_write_kb': 20748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4147}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.986 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:27.987 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'8243", 'reported_seq': 15571, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:25.098916+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:17:25.098916+0000', 'last_peered': '2024-09-11T01:17:25.098916+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:17:25.098916+0000', 'last_undegraded': '2024-09-11T01:17:25.098916+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 8243, 'log_dups_size': 0, 'ondisk_log_size': 8243, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33735139, 'num_objects': 8237, 'num_object_clones': 0, 'num_object_copies': 16474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 498, 'num_objects_unfound': 0, 'num_objects_dirty': 8237, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6110, 'num_write_kb': 24440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3627, 'num_bytes_recovered': 14848483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 498}, {'shards': '2', 'objects': 7739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:27.987 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:27.987 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'8344", 'reported_seq': 8464, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.618530+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:25.618530+0000', 'last_peered': '2024-09-11T01:17:25.618530+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:25.618530+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8344, 'log_dups_size': 0, 'ondisk_log_size': 8344, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34144256, 'num_objects': 8336, 'num_object_clones': 0, 'num_object_copies': 16672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8336, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4708, 'num_write_kb': 18832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4231}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.988 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:27.988 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'8078", 'reported_seq': 8163, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.064007+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:25.064007+0000', 'last_peered': '2024-09-11T01:17:25.064007+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:25.064007+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 8078, 'log_dups_size': 0, 'ondisk_log_size': 8078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33067491, 'num_objects': 8074, 'num_object_clones': 0, 'num_object_copies': 16148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8074, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5847, 'num_write_kb': 23388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 4047}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:27.988 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:27.988 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'8346", 'reported_seq': 8430, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.047253+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:25.047253+0000', 'last_peered': '2024-09-11T01:17:25.047253+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:25.047253+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8346, 'log_dups_size': 0, 'ondisk_log_size': 8346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34160640, 'num_objects': 8340, 'num_object_clones': 0, 'num_object_copies': 16680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5983, 'num_write_kb': 23932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4163}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:27.988 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:27.989 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'8269", 'reported_seq': 8390, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.694398+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:25.694398+0000', 'last_peered': '2024-09-11T01:17:25.694398+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:25.694398+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 8269, 'log_dups_size': 0, 'ondisk_log_size': 8269, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33861632, 'num_objects': 8267, 'num_object_clones': 0, 'num_object_copies': 16534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4959, 'num_write_kb': 19836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4103}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.989 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:27.989 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'8432", 'reported_seq': 8557, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.664357+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:25.664357+0000', 'last_peered': '2024-09-11T01:17:25.664357+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:25.664357+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 8432, 'log_dups_size': 0, 'ondisk_log_size': 8432, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34504704, 'num_objects': 8424, 'num_object_clones': 0, 'num_object_copies': 16848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8424, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4825, 'num_write_kb': 19300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4232}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.989 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:27.989 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'8285", 'reported_seq': 8409, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.582213+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:25.582213+0000', 'last_peered': '2024-09-11T01:17:25.582213+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:25.582213+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 8285, 'log_dups_size': 0, 'ondisk_log_size': 8285, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33918976, 'num_objects': 8281, 'num_object_clones': 0, 'num_object_copies': 16562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4820, 'num_write_kb': 19280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4133}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:27.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'8186", 'reported_seq': 8312, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:25.661792+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:25.661792+0000', 'last_peered': '2024-09-11T01:17:25.661792+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:25.661792+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 8186, 'log_dups_size': 0, 'ondisk_log_size': 8186, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33481670, 'num_objects': 8176, 'num_object_clones': 0, 'num_object_copies': 16352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8176, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4843, 'num_write_kb': 19372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4022}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:27.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:28.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 172 20 133271 133251 3.02556 1.75 0.014812 0.0258127 2024-09-11T01:17:28.735 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:28.736 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/263892 objects degraded (17.245%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":249,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131946,"data_bytes":540406513,"bytes_used":1829277696,"bytes_avail":288081014784,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":263892,"degraded_ratio":0.17244933533415185,"misplaced_objects":498,"misplaced_total":263892,"misplaced_ratio":0.0018871356464008004,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":194108,"recovering_keys_per_sec":0,"num_objects_recovered":569,"num_bytes_recovered":2330624,"num_keys_recovered":0,"write_bytes_sec":2449722,"read_op_per_sec":0,"write_op_per_sec":598},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:28.762 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:28.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:29.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 173 20 133710 133690 3.01798 1.71484 0.031018 0.0258788 2024-09-11T01:17:29.405 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:29.406 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:29.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-11T01:17:28.464403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540406513,"num_objects":131946,"num_object_clones":0,"num_object_copies":263892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":131946,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":91081,"num_write_kb":364318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3629,"num_bytes_recovered":14852598,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132019,"ondisk_log_size":132019,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1802616,"kb_used_data":1073156,"kb_used_omap":12,"kb_used_meta":729395,"kb_avail":281312904,"statfs":{"total":289910292480,"available":288064413696,"internally_reserved":0,"allocated":1098911744,"data_stored":1098089522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":746900487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,4,9,16,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19488768,"num_objects":4758,"num_object_clones":0,"num_object_copies":9516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-394,"num_objects_unfound":0,"num_objects_dirty":4758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4758,"num_write_kb":19032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1613824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006904"},"pg_stats":[{"pgid":"2.f","version":"39'8154","reported_seq":8218,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.648032+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:25.648032+0000","last_peered":"2024-09-11T01:17:25.648032+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:25.648032+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8154,"log_dups_size":0,"ondisk_log_size":8154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33398784,"num_objects":8154,"num_object_clones":0,"num_object_copies":16308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8154,"num_write_kb":32616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4031},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8252","reported_seq":8321,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.093052+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:25.093052+0000","last_peered":"2024-09-11T01:17:25.093052+0000","last_clean":"2024-09-11T01:17:25.093052+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:25.093052+0000","last_undegraded":"2024-09-11T01:17:25.093052+0000","last_fullsized":"2024-09-11T01:17:25.093052+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8252,"log_dups_size":0,"ondisk_log_size":8252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6350,"num_write_kb":25400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8130","reported_seq":8203,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.104597+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:25.104597+0000","last_peered":"2024-09-11T01:17:25.104597+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:25.104597+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8130,"log_dups_size":0,"ondisk_log_size":8130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284096,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6191,"num_write_kb":24764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8192","reported_seq":8228,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.983506+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:24.983506+0000","last_peered":"2024-09-11T01:17:24.983506+0000","last_clean":"2024-09-11T01:17:24.983506+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:24.983506+0000","last_undegraded":"2024-09-11T01:17:24.983506+0000","last_fullsized":"2024-09-11T01:17:24.983506+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8192,"log_dups_size":0,"ondisk_log_size":8192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33554432,"num_objects":8192,"num_object_clones":0,"num_object_copies":16384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8196","reported_seq":8308,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.665627+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:25.665627+0000","last_peered":"2024-09-11T01:17:25.665627+0000","last_clean":"2024-09-11T01:17:25.665627+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:25.665627+0000","last_undegraded":"2024-09-11T01:17:25.665627+0000","last_fullsized":"2024-09-11T01:17:25.665627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8196,"log_dups_size":0,"ondisk_log_size":8196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33542627,"num_objects":8190,"num_object_clones":0,"num_object_copies":16380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4994,"num_write_kb":19976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8224","reported_seq":8349,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.555514+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:25.555514+0000","last_peered":"2024-09-11T01:17:25.555514+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:25.555514+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8224,"log_dups_size":0,"ondisk_log_size":8224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33665507,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4712,"num_write_kb":18848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4105},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8434","reported_seq":8558,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.638568+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:25.638568+0000","last_peered":"2024-09-11T01:17:25.638568+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:25.638568+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8434,"log_dups_size":0,"ondisk_log_size":8434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34522054,"num_objects":8430,"num_object_clones":0,"num_object_copies":16860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8430,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5187,"num_write_kb":20748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4147},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8251","reported_seq":8356,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.611578+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:25.611578+0000","last_peered":"2024-09-11T01:17:25.611578+0000","last_clean":"2024-09-11T01:17:25.611578+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:25.611578+0000","last_undegraded":"2024-09-11T01:17:25.611578+0000","last_fullsized":"2024-09-11T01:17:25.611578+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8251,"log_dups_size":0,"ondisk_log_size":8251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33787904,"num_objects":8249,"num_object_clones":0,"num_object_copies":16498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5204,"num_write_kb":20816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8243","reported_seq":15571,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:25.098916+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:25.098916+0000","last_peered":"2024-09-11T01:17:25.098916+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:25.098916+0000","last_undegraded":"2024-09-11T01:17:25.098916+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8243,"log_dups_size":0,"ondisk_log_size":8243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33735139,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6110,"num_write_kb":24440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3627,"num_bytes_recovered":14848483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":498},{"shards":"2","objects":7739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8344","reported_seq":8464,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.618530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:25.618530+0000","last_peered":"2024-09-11T01:17:25.618530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:25.618530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8344,"log_dups_size":0,"ondisk_log_size":8344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4231},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8078","reported_seq":8163,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.064007+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:25.064007+0000","last_peered":"2024-09-11T01:17:25.064007+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:25.064007+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8078,"log_dups_size":0,"ondisk_log_size":8078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33067491,"num_objects":8074,"num_object_clones":0,"num_object_copies":16148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5847,"num_write_kb":23388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.408712+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.408712+0000","last_peered":"2024-09-11T01:15:20.408712+0000","last_clean":"2024-09-11T01:15:20.408712+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.408712+0000","last_undegraded":"2024-09-11T01:15:20.408712+0000","last_fullsized":"2024-09-11T01:15:20.408712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8346","reported_seq":8430,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.047253+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:25.047253+0000","last_peered":"2024-09-11T01:17:25.047253+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:25.047253+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8346,"log_dups_size":0,"ondisk_log_size":8346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5983,"num_write_kb":23932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4163}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8269","reported_seq":8390,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.694398+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:25.694398+0000","last_peered":"2024-09-11T01:17:25.694398+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:25.694398+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8269,"log_dups_size":0,"ondisk_log_size":8269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33861632,"num_objects":8267,"num_object_clones":0,"num_object_copies":16534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4959,"num_write_kb":19836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4103},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.481918+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.481918+0000","last_peered":"2024-09-11T01:15:20.481918+0000","last_clean":"2024-09-11T01:15:20.481918+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.481918+0000","last_undegraded":"2024-09-11T01:15:20.481918+0000","last_fullsized":"2024-09-11T01:15:20.481918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8432","reported_seq":8557,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.664357+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:25.664357+0000","last_peered":"2024-09-11T01:17:25.664357+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:25.664357+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8432,"log_dups_size":0,"ondisk_log_size":8432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34504704,"num_objects":8424,"num_object_clones":0,"num_object_copies":16848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4825,"num_write_kb":19300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4232},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8285","reported_seq":8409,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.582213+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:25.582213+0000","last_peered":"2024-09-11T01:17:25.582213+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:25.582213+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8285,"log_dups_size":0,"ondisk_log_size":8285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33918976,"num_objects":8281,"num_object_clones":0,"num_object_copies":16562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4820,"num_write_kb":19280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4133},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.507048+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.507048+0000","last_peered":"2024-09-11T01:15:20.507048+0000","last_clean":"2024-09-11T01:15:20.507048+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.507048+0000","last_undegraded":"2024-09-11T01:15:20.507048+0000","last_fullsized":"2024-09-11T01:15:20.507048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.997186+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.997186+0000","last_peered":"2024-09-11T01:15:19.997186+0000","last_clean":"2024-09-11T01:15:19.997186+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.997186+0000","last_undegraded":"2024-09-11T01:15:19.997186+0000","last_fullsized":"2024-09-11T01:15:19.997186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8186","reported_seq":8312,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.661792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:25.661792+0000","last_peered":"2024-09-11T01:17:25.661792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:25.661792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8186,"log_dups_size":0,"ondisk_log_size":8186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33481670,"num_objects":8176,"num_object_clones":0,"num_object_copies":16352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4843,"num_write_kb":19372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4022},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":540406494,"num_objects":131944,"num_object_clones":0,"num_object_copies":263888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":498,"num_objects_unfound":0,"num_objects_dirty":131944,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":91079,"num_write_kb":364316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3628,"num_bytes_recovered":14852579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1100222464,"data_stored":1100070718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132016,"ondisk_log_size":132016,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":625692,"kb_used_data":395464,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93746148,"statfs":{"total":96636764160,"available":95996055552,"internally_reserved":0,"allocated":404955136,"data_stored":404698232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":235728558},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422732,"kb_used_data":216124,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93949108,"statfs":{"total":96636764160,"available":96203886592,"internally_reserved":0,"allocated":221310976,"data_stored":221006639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":211545775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754192,"kb_used_data":461568,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93617648,"statfs":{"total":96636764160,"available":95864471552,"internally_reserved":0,"allocated":472645632,"data_stored":472384651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":473247744,"data_stored":473211614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221896704,"data_stored":221817218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405078016,"data_stored":405041886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:30.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 174 20 134189 134169 3.01139 1.87109 0.0542875 0.0259357 2024-09-11T01:17:31.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 175 20 134676 134656 3.00505 1.90234 0.0704187 0.0259894 2024-09-11T01:17:32.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 176 20 135164 135144 2.99881 1.90625 0.0238505 0.0260439 2024-09-11T01:17:32.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:32.997 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:32.997 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:33.022 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-11T01:17:32.468077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549872369,"num_objects":134257,"num_object_clones":0,"num_object_copies":268514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":134257,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":93392,"num_write_kb":373562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3835,"num_bytes_recovered":15696374,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134330,"ondisk_log_size":134330,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1822664,"kb_used_data":1084888,"kb_used_omap":12,"kb_used_meta":737715,"kb_avail":281292856,"statfs":{"total":289910292480,"available":288043884544,"internally_reserved":0,"allocated":1110925312,"data_stored":1110103090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":755420167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,11,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25210880,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005764"},"pg_stats":[{"pgid":"2.f","version":"39'8292","reported_seq":8356,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.688993+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:30.688993+0000","last_peered":"2024-09-11T01:17:30.688993+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:30.688993+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8292,"log_dups_size":0,"ondisk_log_size":8292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964032,"num_objects":8292,"num_object_clones":0,"num_object_copies":16584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8292,"num_write_kb":33168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4169},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8390","reported_seq":8461,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.054990+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:30.054990+0000","last_peered":"2024-09-11T01:17:30.054990+0000","last_clean":"2024-09-11T01:17:30.054990+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:30.054990+0000","last_undegraded":"2024-09-11T01:17:30.054990+0000","last_fullsized":"2024-09-11T01:17:30.054990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34341830,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8254","reported_seq":8328,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.027949+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:30.027949+0000","last_peered":"2024-09-11T01:17:30.027949+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:30.027949+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8254,"log_dups_size":0,"ondisk_log_size":8254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33792000,"num_objects":8250,"num_object_clones":0,"num_object_copies":16500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8324","reported_seq":8360,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.045083+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:30.045083+0000","last_peered":"2024-09-11T01:17:30.045083+0000","last_clean":"2024-09-11T01:17:30.045083+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:30.045083+0000","last_undegraded":"2024-09-11T01:17:30.045083+0000","last_fullsized":"2024-09-11T01:17:30.045083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8324,"log_dups_size":0,"ondisk_log_size":8324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8324,"num_write_kb":33296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8332","reported_seq":8444,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.622881+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:30.622881+0000","last_peered":"2024-09-11T01:17:30.622881+0000","last_clean":"2024-09-11T01:17:30.622881+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:30.622881+0000","last_undegraded":"2024-09-11T01:17:30.622881+0000","last_fullsized":"2024-09-11T01:17:30.622881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8332,"log_dups_size":0,"ondisk_log_size":8332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34099683,"num_objects":8326,"num_object_clones":0,"num_object_copies":16652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5130,"num_write_kb":20520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8374","reported_seq":8498,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.655153+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:30.655153+0000","last_peered":"2024-09-11T01:17:30.655153+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:30.655153+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279907,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4862,"num_write_kb":19448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8587","reported_seq":8711,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.678683+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:30.678683+0000","last_peered":"2024-09-11T01:17:30.678683+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:30.678683+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35148742,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5340,"num_write_kb":21360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8411","reported_seq":8521,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.628927+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:30.628927+0000","last_peered":"2024-09-11T01:17:30.628927+0000","last_clean":"2024-09-11T01:17:30.628927+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:30.628927+0000","last_undegraded":"2024-09-11T01:17:30.628927+0000","last_fullsized":"2024-09-11T01:17:30.628927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8411,"log_dups_size":0,"ondisk_log_size":8411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5364,"num_write_kb":21456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8370","reported_seq":16110,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:30.113706+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:30.113706+0000","last_peered":"2024-09-11T01:17:30.113706+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:30.113706+0000","last_undegraded":"2024-09-11T01:17:30.113706+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8370,"log_dups_size":0,"ondisk_log_size":8370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34255331,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3833,"num_bytes_recovered":15692259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":292},{"shards":"2","objects":8072}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8493","reported_seq":8613,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.637582+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:30.637582+0000","last_peered":"2024-09-11T01:17:30.637582+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:30.637582+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8493,"log_dups_size":0,"ondisk_log_size":8493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4857,"num_write_kb":19428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8212","reported_seq":8297,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.008068+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:30.008068+0000","last_peered":"2024-09-11T01:17:30.008068+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:30.008068+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8212,"log_dups_size":0,"ondisk_log_size":8212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33616355,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5981,"num_write_kb":23924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4181}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.408712+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.408712+0000","last_peered":"2024-09-11T01:15:20.408712+0000","last_clean":"2024-09-11T01:15:20.408712+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.408712+0000","last_undegraded":"2024-09-11T01:15:20.408712+0000","last_fullsized":"2024-09-11T01:15:20.408712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8498","reported_seq":8580,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.063167+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:30.063167+0000","last_peered":"2024-09-11T01:17:30.063167+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:30.063167+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8430","reported_seq":8549,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.619411+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:30.619411+0000","last_peered":"2024-09-11T01:17:30.619411+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:30.619411+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8430,"log_dups_size":0,"ondisk_log_size":8430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4264},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.481918+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.481918+0000","last_peered":"2024-09-11T01:15:20.481918+0000","last_clean":"2024-09-11T01:15:20.481918+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.481918+0000","last_undegraded":"2024-09-11T01:15:20.481918+0000","last_fullsized":"2024-09-11T01:15:20.481918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8580","reported_seq":8705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.555396+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:30.555396+0000","last_peered":"2024-09-11T01:17:30.555396+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:30.555396+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8580,"log_dups_size":0,"ondisk_log_size":8580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35110912,"num_objects":8572,"num_object_clones":0,"num_object_copies":17144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4973,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8446","reported_seq":8570,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.647007+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:30.647007+0000","last_peered":"2024-09-11T01:17:30.647007+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:30.647007+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8446,"log_dups_size":0,"ondisk_log_size":8446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4981,"num_write_kb":19924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4294},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.507048+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.507048+0000","last_peered":"2024-09-11T01:15:20.507048+0000","last_clean":"2024-09-11T01:15:20.507048+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.507048+0000","last_undegraded":"2024-09-11T01:15:20.507048+0000","last_fullsized":"2024-09-11T01:15:20.507048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.997186+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.997186+0000","last_peered":"2024-09-11T01:15:19.997186+0000","last_clean":"2024-09-11T01:15:19.997186+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.997186+0000","last_undegraded":"2024-09-11T01:15:19.997186+0000","last_fullsized":"2024-09-11T01:15:19.997186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8334","reported_seq":8460,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.676801+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:30.676801+0000","last_peered":"2024-09-11T01:17:30.676801+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:30.676801+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087878,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4991,"num_write_kb":19964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4170},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549872350,"num_objects":134255,"num_object_clones":0,"num_object_copies":268510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":134255,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":93390,"num_write_kb":373560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":15696355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1112125440,"data_stored":1111973694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134327,"ondisk_log_size":134327,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634424,"kb_used_data":400040,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93737416,"statfs":{"total":96636764160,"available":95987113984,"internally_reserved":0,"allocated":409640960,"data_stored":409384056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422732,"kb_used_data":216124,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93949108,"statfs":{"total":96636764160,"available":96203886592,"internally_reserved":0,"allocated":221310976,"data_stored":221006639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":211545775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765508,"kb_used_data":468724,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93606332,"statfs":{"total":96636764160,"available":95852883968,"internally_reserved":0,"allocated":479973376,"data_stored":479712395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":303885994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480559104,"data_stored":480522974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221896704,"data_stored":221817218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409669632,"data_stored":409633502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:33.023 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:33.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 177 20 135606 135586 2.99162 1.72656 0.0231449 0.0261052 2024-09-11T01:17:33.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:33.598 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:33.626 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-11T01:17:32.468077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549872369,"num_objects":134257,"num_object_clones":0,"num_object_copies":268514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":134257,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":93392,"num_write_kb":373562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3835,"num_bytes_recovered":15696374,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134330,"ondisk_log_size":134330,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1822664,"kb_used_data":1084888,"kb_used_omap":12,"kb_used_meta":737715,"kb_avail":281292856,"statfs":{"total":289910292480,"available":288043884544,"internally_reserved":0,"allocated":1110925312,"data_stored":1110103090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":755420167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,11,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25210880,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005764"},"pg_stats":[{"pgid":"2.f","version":"39'8292","reported_seq":8356,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.688993+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:30.688993+0000","last_peered":"2024-09-11T01:17:30.688993+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:30.688993+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8292,"log_dups_size":0,"ondisk_log_size":8292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964032,"num_objects":8292,"num_object_clones":0,"num_object_copies":16584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8292,"num_write_kb":33168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4169},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8390","reported_seq":8461,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.054990+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:30.054990+0000","last_peered":"2024-09-11T01:17:30.054990+0000","last_clean":"2024-09-11T01:17:30.054990+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:30.054990+0000","last_undegraded":"2024-09-11T01:17:30.054990+0000","last_fullsized":"2024-09-11T01:17:30.054990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34341830,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8254","reported_seq":8328,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.027949+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:30.027949+0000","last_peered":"2024-09-11T01:17:30.027949+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:30.027949+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8254,"log_dups_size":0,"ondisk_log_size":8254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33792000,"num_objects":8250,"num_object_clones":0,"num_object_copies":16500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8324","reported_seq":8360,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.045083+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:30.045083+0000","last_peered":"2024-09-11T01:17:30.045083+0000","last_clean":"2024-09-11T01:17:30.045083+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:30.045083+0000","last_undegraded":"2024-09-11T01:17:30.045083+0000","last_fullsized":"2024-09-11T01:17:30.045083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8324,"log_dups_size":0,"ondisk_log_size":8324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8324,"num_write_kb":33296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8332","reported_seq":8444,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.622881+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:30.622881+0000","last_peered":"2024-09-11T01:17:30.622881+0000","last_clean":"2024-09-11T01:17:30.622881+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:30.622881+0000","last_undegraded":"2024-09-11T01:17:30.622881+0000","last_fullsized":"2024-09-11T01:17:30.622881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8332,"log_dups_size":0,"ondisk_log_size":8332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34099683,"num_objects":8326,"num_object_clones":0,"num_object_copies":16652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5130,"num_write_kb":20520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8374","reported_seq":8498,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.655153+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:30.655153+0000","last_peered":"2024-09-11T01:17:30.655153+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:30.655153+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279907,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4862,"num_write_kb":19448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8587","reported_seq":8711,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.678683+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:30.678683+0000","last_peered":"2024-09-11T01:17:30.678683+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:30.678683+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35148742,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5340,"num_write_kb":21360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8411","reported_seq":8521,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.628927+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:30.628927+0000","last_peered":"2024-09-11T01:17:30.628927+0000","last_clean":"2024-09-11T01:17:30.628927+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:30.628927+0000","last_undegraded":"2024-09-11T01:17:30.628927+0000","last_fullsized":"2024-09-11T01:17:30.628927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8411,"log_dups_size":0,"ondisk_log_size":8411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5364,"num_write_kb":21456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8370","reported_seq":16110,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:30.113706+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:30.113706+0000","last_peered":"2024-09-11T01:17:30.113706+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:30.113706+0000","last_undegraded":"2024-09-11T01:17:30.113706+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8370,"log_dups_size":0,"ondisk_log_size":8370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34255331,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3833,"num_bytes_recovered":15692259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":292},{"shards":"2","objects":8072}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8493","reported_seq":8613,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.637582+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:30.637582+0000","last_peered":"2024-09-11T01:17:30.637582+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:30.637582+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8493,"log_dups_size":0,"ondisk_log_size":8493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4857,"num_write_kb":19428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8212","reported_seq":8297,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.008068+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:30.008068+0000","last_peered":"2024-09-11T01:17:30.008068+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:30.008068+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8212,"log_dups_size":0,"ondisk_log_size":8212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33616355,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5981,"num_write_kb":23924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4181}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.408712+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.408712+0000","last_peered":"2024-09-11T01:15:20.408712+0000","last_clean":"2024-09-11T01:15:20.408712+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.408712+0000","last_undegraded":"2024-09-11T01:15:20.408712+0000","last_fullsized":"2024-09-11T01:15:20.408712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8498","reported_seq":8580,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.063167+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:30.063167+0000","last_peered":"2024-09-11T01:17:30.063167+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:30.063167+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8430","reported_seq":8549,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.619411+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:30.619411+0000","last_peered":"2024-09-11T01:17:30.619411+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:30.619411+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8430,"log_dups_size":0,"ondisk_log_size":8430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4264},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.481918+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.481918+0000","last_peered":"2024-09-11T01:15:20.481918+0000","last_clean":"2024-09-11T01:15:20.481918+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.481918+0000","last_undegraded":"2024-09-11T01:15:20.481918+0000","last_fullsized":"2024-09-11T01:15:20.481918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8580","reported_seq":8705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.555396+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:30.555396+0000","last_peered":"2024-09-11T01:17:30.555396+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:30.555396+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8580,"log_dups_size":0,"ondisk_log_size":8580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35110912,"num_objects":8572,"num_object_clones":0,"num_object_copies":17144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4973,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8446","reported_seq":8570,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.647007+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:30.647007+0000","last_peered":"2024-09-11T01:17:30.647007+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:30.647007+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8446,"log_dups_size":0,"ondisk_log_size":8446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4981,"num_write_kb":19924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4294},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.507048+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.507048+0000","last_peered":"2024-09-11T01:15:20.507048+0000","last_clean":"2024-09-11T01:15:20.507048+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.507048+0000","last_undegraded":"2024-09-11T01:15:20.507048+0000","last_fullsized":"2024-09-11T01:15:20.507048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.997186+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.997186+0000","last_peered":"2024-09-11T01:15:19.997186+0000","last_clean":"2024-09-11T01:15:19.997186+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.997186+0000","last_undegraded":"2024-09-11T01:15:19.997186+0000","last_fullsized":"2024-09-11T01:15:19.997186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8334","reported_seq":8460,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.676801+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:30.676801+0000","last_peered":"2024-09-11T01:17:30.676801+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:30.676801+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087878,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4991,"num_write_kb":19964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4170},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549872350,"num_objects":134255,"num_object_clones":0,"num_object_copies":268510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":134255,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":93390,"num_write_kb":373560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":15696355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1112125440,"data_stored":1111973694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134327,"ondisk_log_size":134327,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634424,"kb_used_data":400040,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93737416,"statfs":{"total":96636764160,"available":95987113984,"internally_reserved":0,"allocated":409640960,"data_stored":409384056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789979,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422732,"kb_used_data":216124,"kb_used_omap":4,"kb_used_meta":206587,"kb_avail":93949108,"statfs":{"total":96636764160,"available":96203886592,"internally_reserved":0,"allocated":221310976,"data_stored":221006639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":211545775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765508,"kb_used_data":468724,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93606332,"statfs":{"total":96636764160,"available":95852883968,"internally_reserved":0,"allocated":479973376,"data_stored":479712395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":303885994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480559104,"data_stored":480522974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221896704,"data_stored":221817218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409669632,"data_stored":409633502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'8292", 'reported_seq': 8356, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.688993+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:30.688993+0000', 'last_peered': '2024-09-11T01:17:30.688993+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:30.688993+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 8292, 'log_dups_size': 0, 'ondisk_log_size': 8292, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33964032, 'num_objects': 8292, 'num_object_clones': 0, 'num_object_copies': 16584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8292, 'num_write_kb': 33168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4169}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'8254", 'reported_seq': 8328, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.027949+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:30.027949+0000', 'last_peered': '2024-09-11T01:17:30.027949+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:30.027949+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 8254, 'log_dups_size': 0, 'ondisk_log_size': 8254, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33792000, 'num_objects': 8250, 'num_object_clones': 0, 'num_object_copies': 16500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8250, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6315, 'num_write_kb': 25260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 4245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'8374", 'reported_seq': 8498, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.655153+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:30.655153+0000', 'last_peered': '2024-09-11T01:17:30.655153+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:30.655153+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 8374, 'log_dups_size': 0, 'ondisk_log_size': 8374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34279907, 'num_objects': 8370, 'num_object_clones': 0, 'num_object_copies': 16740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8370, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4862, 'num_write_kb': 19448, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4255}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:33.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'8587", 'reported_seq': 8711, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.678683+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:30.678683+0000', 'last_peered': '2024-09-11T01:17:30.678683+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:30.678683+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 8587, 'log_dups_size': 0, 'ondisk_log_size': 8587, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35148742, 'num_objects': 8583, 'num_object_clones': 0, 'num_object_copies': 17166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8583, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5340, 'num_write_kb': 21360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4300}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "39'8370", 'reported_seq': 16110, 'reported_epoch': 39, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:30.113706+0000', 'last_change': '2024-09-11T01:15:20.443442+0000', 'last_active': '2024-09-11T01:17:30.113706+0000', 'last_peered': '2024-09-11T01:17:30.113706+0000', 'last_clean': '2024-09-11T01:15:17.939231+0000', 'last_became_active': '2024-09-11T01:15:20.419947+0000', 'last_became_peered': '2024-09-11T01:15:20.419947+0000', 'last_unstale': '2024-09-11T01:17:30.113706+0000', 'last_undegraded': '2024-09-11T01:17:30.113706+0000', 'last_fullsized': '2024-09-11T01:15:19.991693+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2163", 'last_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:51.929170+0000', 'objects_scrubbed': 2127, 'log_size': 8370, 'log_dups_size': 0, 'ondisk_log_size': 8370, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 703, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34255331, 'num_objects': 8364, 'num_object_clones': 0, 'num_object_copies': 16728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 292, 'num_objects_unfound': 0, 'num_objects_dirty': 8364, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6237, 'num_write_kb': 24948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3833, 'num_bytes_recovered': 15692259, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 292}, {'shards': '2', 'objects': 8072}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'8493", 'reported_seq': 8613, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.637582+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:30.637582+0000', 'last_peered': '2024-09-11T01:17:30.637582+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:30.637582+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8493, 'log_dups_size': 0, 'ondisk_log_size': 8493, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34754560, 'num_objects': 8485, 'num_object_clones': 0, 'num_object_copies': 16970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4857, 'num_write_kb': 19428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4380}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'8212", 'reported_seq': 8297, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.008068+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:30.008068+0000', 'last_peered': '2024-09-11T01:17:30.008068+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:30.008068+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 8212, 'log_dups_size': 0, 'ondisk_log_size': 8212, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33616355, 'num_objects': 8208, 'num_object_clones': 0, 'num_object_copies': 16416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8208, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5981, 'num_write_kb': 23924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 4181}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:33.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'8498", 'reported_seq': 8580, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.063167+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:30.063167+0000', 'last_peered': '2024-09-11T01:17:30.063167+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:30.063167+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8498, 'log_dups_size': 0, 'ondisk_log_size': 8498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34783232, 'num_objects': 8492, 'num_object_clones': 0, 'num_object_copies': 16984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8492, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6135, 'num_write_kb': 24540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4315}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'8430", 'reported_seq': 8549, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.619411+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:30.619411+0000', 'last_peered': '2024-09-11T01:17:30.619411+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:30.619411+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 8430, 'log_dups_size': 0, 'ondisk_log_size': 8430, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34521088, 'num_objects': 8428, 'num_object_clones': 0, 'num_object_copies': 16856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5120, 'num_write_kb': 20480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4264}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'8580", 'reported_seq': 8705, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.555396+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:30.555396+0000', 'last_peered': '2024-09-11T01:17:30.555396+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:30.555396+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 8580, 'log_dups_size': 0, 'ondisk_log_size': 8580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35110912, 'num_objects': 8572, 'num_object_clones': 0, 'num_object_copies': 17144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4973, 'num_write_kb': 19892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4380}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'8446", 'reported_seq': 8570, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.647007+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:30.647007+0000', 'last_peered': '2024-09-11T01:17:30.647007+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:30.647007+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 8446, 'log_dups_size': 0, 'ondisk_log_size': 8446, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34578432, 'num_objects': 8442, 'num_object_clones': 0, 'num_object_copies': 16884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8442, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4981, 'num_write_kb': 19924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4294}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:33.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'8334", 'reported_seq': 8460, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:30.676801+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:30.676801+0000', 'last_peered': '2024-09-11T01:17:30.676801+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:30.676801+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 8334, 'log_dups_size': 0, 'ondisk_log_size': 8334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34087878, 'num_objects': 8324, 'num_object_clones': 0, 'num_object_copies': 16648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8324, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4991, 'num_write_kb': 19964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4170}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:33.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:34.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:34.339 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/268514 objects degraded (16.948%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":254,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134257,"data_bytes":549872369,"bytes_used":1866407936,"bytes_avail":288043884544,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":268514,"degraded_ratio":0.16948092092032446,"misplaced_objects":292,"misplaced_total":268514,"misplaced_ratio":0.001087466575299612,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":141244,"recovering_keys_per_sec":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"write_bytes_sec":2099897,"read_op_per_sec":0,"write_op_per_sec":512},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:34.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 178 20 136048 136028 2.98451 1.72656 0.015539 0.0261677 2024-09-11T01:17:34.366 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:34.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:34.949 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:34.951 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:34.980 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-11T01:17:34.468676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549872369,"num_objects":134257,"num_object_clones":0,"num_object_copies":268514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":134257,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":93392,"num_write_kb":373562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3835,"num_bytes_recovered":15696374,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134330,"ondisk_log_size":134330,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1836340,"kb_used_data":1092996,"kb_used_omap":12,"kb_used_meta":743283,"kb_avail":281279180,"statfs":{"total":289910292480,"available":288029880320,"internally_reserved":0,"allocated":1119227904,"data_stored":1118405682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":761121799},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,18,7,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19161088,"num_objects":4678,"num_object_clones":0,"num_object_copies":9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":4678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4678,"num_write_kb":18712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005854"},"pg_stats":[{"pgid":"2.f","version":"39'8292","reported_seq":8356,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.688993+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:30.688993+0000","last_peered":"2024-09-11T01:17:30.688993+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:30.688993+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8292,"log_dups_size":0,"ondisk_log_size":8292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964032,"num_objects":8292,"num_object_clones":0,"num_object_copies":16584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8292,"num_write_kb":33168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4169},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8390","reported_seq":8461,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.054990+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:30.054990+0000","last_peered":"2024-09-11T01:17:30.054990+0000","last_clean":"2024-09-11T01:17:30.054990+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:30.054990+0000","last_undegraded":"2024-09-11T01:17:30.054990+0000","last_fullsized":"2024-09-11T01:17:30.054990+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8390,"log_dups_size":0,"ondisk_log_size":8390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34341830,"num_objects":8386,"num_object_clones":0,"num_object_copies":16772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8386,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8254","reported_seq":8328,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.027949+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:30.027949+0000","last_peered":"2024-09-11T01:17:30.027949+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:30.027949+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8254,"log_dups_size":0,"ondisk_log_size":8254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33792000,"num_objects":8250,"num_object_clones":0,"num_object_copies":16500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8324","reported_seq":8360,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.045083+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:30.045083+0000","last_peered":"2024-09-11T01:17:30.045083+0000","last_clean":"2024-09-11T01:17:30.045083+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:30.045083+0000","last_undegraded":"2024-09-11T01:17:30.045083+0000","last_fullsized":"2024-09-11T01:17:30.045083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8324,"log_dups_size":0,"ondisk_log_size":8324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8324,"num_write_kb":33296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8332","reported_seq":8444,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.622881+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:30.622881+0000","last_peered":"2024-09-11T01:17:30.622881+0000","last_clean":"2024-09-11T01:17:30.622881+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:30.622881+0000","last_undegraded":"2024-09-11T01:17:30.622881+0000","last_fullsized":"2024-09-11T01:17:30.622881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8332,"log_dups_size":0,"ondisk_log_size":8332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34099683,"num_objects":8326,"num_object_clones":0,"num_object_copies":16652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5130,"num_write_kb":20520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8374","reported_seq":8498,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.655153+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:30.655153+0000","last_peered":"2024-09-11T01:17:30.655153+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:30.655153+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8374,"log_dups_size":0,"ondisk_log_size":8374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34279907,"num_objects":8370,"num_object_clones":0,"num_object_copies":16740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4862,"num_write_kb":19448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8587","reported_seq":8711,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.678683+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:30.678683+0000","last_peered":"2024-09-11T01:17:30.678683+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:30.678683+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35148742,"num_objects":8583,"num_object_clones":0,"num_object_copies":17166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8583,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5340,"num_write_kb":21360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8411","reported_seq":8521,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:30.628927+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:30.628927+0000","last_peered":"2024-09-11T01:17:30.628927+0000","last_clean":"2024-09-11T01:17:30.628927+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:30.628927+0000","last_undegraded":"2024-09-11T01:17:30.628927+0000","last_fullsized":"2024-09-11T01:17:30.628927+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8411,"log_dups_size":0,"ondisk_log_size":8411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5364,"num_write_kb":21456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.994329+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:15:19.994329+0000","last_peered":"2024-09-11T01:15:19.994329+0000","last_clean":"2024-09-11T01:15:19.994329+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:15:19.994329+0000","last_undegraded":"2024-09-11T01:15:19.994329+0000","last_fullsized":"2024-09-11T01:15:19.994329+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"39'8370","reported_seq":16110,"reported_epoch":39,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:30.113706+0000","last_change":"2024-09-11T01:15:20.443442+0000","last_active":"2024-09-11T01:17:30.113706+0000","last_peered":"2024-09-11T01:17:30.113706+0000","last_clean":"2024-09-11T01:15:17.939231+0000","last_became_active":"2024-09-11T01:15:20.419947+0000","last_became_peered":"2024-09-11T01:15:20.419947+0000","last_unstale":"2024-09-11T01:17:30.113706+0000","last_undegraded":"2024-09-11T01:17:30.113706+0000","last_fullsized":"2024-09-11T01:15:19.991693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8370,"log_dups_size":0,"ondisk_log_size":8370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34255331,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6237,"num_write_kb":24948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3833,"num_bytes_recovered":15692259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":292},{"shards":"2","objects":8072}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8493","reported_seq":8613,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.637582+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:30.637582+0000","last_peered":"2024-09-11T01:17:30.637582+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:30.637582+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8493,"log_dups_size":0,"ondisk_log_size":8493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34754560,"num_objects":8485,"num_object_clones":0,"num_object_copies":16970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4857,"num_write_kb":19428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8212","reported_seq":8297,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.008068+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:30.008068+0000","last_peered":"2024-09-11T01:17:30.008068+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:30.008068+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8212,"log_dups_size":0,"ondisk_log_size":8212,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33616355,"num_objects":8208,"num_object_clones":0,"num_object_copies":16416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5981,"num_write_kb":23924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4181}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.408712+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:15:20.408712+0000","last_peered":"2024-09-11T01:15:20.408712+0000","last_clean":"2024-09-11T01:15:20.408712+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:15:20.408712+0000","last_undegraded":"2024-09-11T01:15:20.408712+0000","last_fullsized":"2024-09-11T01:15:20.408712+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8498","reported_seq":8580,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.063167+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:30.063167+0000","last_peered":"2024-09-11T01:17:30.063167+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:30.063167+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8498,"log_dups_size":0,"ondisk_log_size":8498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34783232,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8430","reported_seq":8549,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.619411+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:30.619411+0000","last_peered":"2024-09-11T01:17:30.619411+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:30.619411+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8430,"log_dups_size":0,"ondisk_log_size":8430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34521088,"num_objects":8428,"num_object_clones":0,"num_object_copies":16856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5120,"num_write_kb":20480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4264},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.481918+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:15:20.481918+0000","last_peered":"2024-09-11T01:15:20.481918+0000","last_clean":"2024-09-11T01:15:20.481918+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:15:20.481918+0000","last_undegraded":"2024-09-11T01:15:20.481918+0000","last_fullsized":"2024-09-11T01:15:20.481918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8580","reported_seq":8705,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.555396+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:30.555396+0000","last_peered":"2024-09-11T01:17:30.555396+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:30.555396+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8580,"log_dups_size":0,"ondisk_log_size":8580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35110912,"num_objects":8572,"num_object_clones":0,"num_object_copies":17144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4973,"num_write_kb":19892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4380},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8446","reported_seq":8570,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.647007+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:30.647007+0000","last_peered":"2024-09-11T01:17:30.647007+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:30.647007+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8446,"log_dups_size":0,"ondisk_log_size":8446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4981,"num_write_kb":19924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4294},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.507048+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:15:20.507048+0000","last_peered":"2024-09-11T01:15:20.507048+0000","last_clean":"2024-09-11T01:15:20.507048+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:15:20.507048+0000","last_undegraded":"2024-09-11T01:15:20.507048+0000","last_fullsized":"2024-09-11T01:15:20.507048+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.997186+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:15:19.997186+0000","last_peered":"2024-09-11T01:15:19.997186+0000","last_clean":"2024-09-11T01:15:19.997186+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:15:19.997186+0000","last_undegraded":"2024-09-11T01:15:19.997186+0000","last_fullsized":"2024-09-11T01:15:19.997186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8334","reported_seq":8460,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:30.676801+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:30.676801+0000","last_peered":"2024-09-11T01:17:30.676801+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:30.676801+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8334,"log_dups_size":0,"ondisk_log_size":8334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087878,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4991,"num_write_kb":19964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4170},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549872350,"num_objects":134255,"num_object_clones":0,"num_object_copies":268510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":292,"num_objects_unfound":0,"num_objects_dirty":134255,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":93390,"num_write_kb":373560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3834,"num_bytes_recovered":15696355,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1120370688,"data_stored":1120218942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134327,"ondisk_log_size":134327,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634424,"kb_used_data":400040,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93737416,"statfs":{"total":96636764160,"available":95987113984,"internally_reserved":0,"allocated":409640960,"data_stored":409384056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":239988398},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789980,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436408,"kb_used_data":224232,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93935432,"statfs":{"total":96636764160,"available":96189882368,"internally_reserved":0,"allocated":229613568,"data_stored":229309231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,11,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765508,"kb_used_data":468724,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93606332,"statfs":{"total":96636764160,"available":95852883968,"internally_reserved":0,"allocated":479973376,"data_stored":479712395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":303885994},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":480559104,"data_stored":480522974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230141952,"data_stored":230062466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409669632,"data_stored":409633502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:35.350 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 179 20 136494 136474 2.97757 1.74219 0.0789486 0.0262298 2024-09-11T01:17:36.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:17:36.350351+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0262864 2024-09-11T01:17:36.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:17:36.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 180 20 136960 136940 2.97114 1.82031 0.0653373 0.0262864 2024-09-11T01:17:37.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 181 20 137438 137418 2.96504 1.86719 0.0171892 0.0263249 2024-09-11T01:17:37.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:38.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 182 20 137724 137704 2.95489 1.11719 0.0200847 0.0264302 2024-09-11T01:17:38.558 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:38.561 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:38.588 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-11T01:17:38.471080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":559710961,"num_objects":136659,"num_object_clones":0,"num_object_copies":273318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136659,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":95794,"num_write_kb":383170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4127,"num_bytes_recovered":16892406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136732,"ondisk_log_size":136732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1869356,"kb_used_data":1112120,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281246164,"statfs":{"total":289910292480,"available":287996071936,"internally_reserved":0,"allocated":1138810880,"data_stored":1137983468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":775343111},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,7,4,6,20,7,1,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-292,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3906,"num_write_kb":15624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003577"},"pg_stats":[{"pgid":"2.f","version":"39'8451","reported_seq":8515,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.698765+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:35.698765+0000","last_peered":"2024-09-11T01:17:35.698765+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:35.698765+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8451,"log_dups_size":0,"ondisk_log_size":8451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615296,"num_objects":8451,"num_object_clones":0,"num_object_copies":16902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8451,"num_write_kb":33804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4328},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8549","reported_seq":8620,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.967804+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:34.967804+0000","last_peered":"2024-09-11T01:17:34.967804+0000","last_clean":"2024-09-11T01:17:34.967804+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:34.967804+0000","last_undegraded":"2024-09-11T01:17:34.967804+0000","last_fullsized":"2024-09-11T01:17:34.967804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8549,"log_dups_size":0,"ondisk_log_size":8549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34993094,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6647,"num_write_kb":26588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8393","reported_seq":8463,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.956861+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:34.956861+0000","last_peered":"2024-09-11T01:17:34.956861+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:34.956861+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8393,"log_dups_size":0,"ondisk_log_size":8393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34361344,"num_objects":8389,"num_object_clones":0,"num_object_copies":16778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8469","reported_seq":8505,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.110790+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:35.110790+0000","last_peered":"2024-09-11T01:17:35.110790+0000","last_clean":"2024-09-11T01:17:35.110790+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:35.110790+0000","last_undegraded":"2024-09-11T01:17:35.110790+0000","last_fullsized":"2024-09-11T01:17:35.110790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8469,"num_write_kb":33876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8469","reported_seq":8581,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.714776+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:35.714776+0000","last_peered":"2024-09-11T01:17:35.714776+0000","last_clean":"2024-09-11T01:17:35.714776+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:35.714776+0000","last_undegraded":"2024-09-11T01:17:35.714776+0000","last_fullsized":"2024-09-11T01:17:35.714776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660835,"num_objects":8463,"num_object_clones":0,"num_object_copies":16926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5267,"num_write_kb":21068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8511","reported_seq":8636,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.685940+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:35.685940+0000","last_peered":"2024-09-11T01:17:35.685940+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:35.685940+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34841059,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4392},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8735","reported_seq":8858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.737508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:35.737508+0000","last_peered":"2024-09-11T01:17:35.737508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:35.737508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35754950,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5488,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8553","reported_seq":8663,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.650462+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:35.650462+0000","last_peered":"2024-09-11T01:17:35.650462+0000","last_clean":"2024-09-11T01:17:35.650462+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:35.650462+0000","last_undegraded":"2024-09-11T01:17:35.650462+0000","last_fullsized":"2024-09-11T01:17:35.650462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35024896,"num_objects":8551,"num_object_clones":0,"num_object_copies":17102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5506,"num_write_kb":22024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8587","reported_seq":16924,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:38.360037+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:38.360037+0000","last_peered":"2024-09-11T01:17:38.360037+0000","last_clean":"2024-09-11T01:17:38.360037+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:38.360037+0000","last_undegraded":"2024-09-11T01:17:38.360037+0000","last_fullsized":"2024-09-11T01:17:38.360037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8647","reported_seq":8766,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.645841+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:35.645841+0000","last_peered":"2024-09-11T01:17:35.645841+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:35.645841+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8647,"log_dups_size":0,"ondisk_log_size":8647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35385344,"num_objects":8639,"num_object_clones":0,"num_object_copies":17278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5011,"num_write_kb":20044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4534},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8355","reported_seq":8440,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.091361+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:35.091361+0000","last_peered":"2024-09-11T01:17:35.091361+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:35.091361+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8355,"log_dups_size":0,"ondisk_log_size":8355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34202083,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6124,"num_write_kb":24496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8643","reported_seq":8727,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.071832+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:35.071832+0000","last_peered":"2024-09-11T01:17:35.071832+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:35.071832+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8643,"log_dups_size":0,"ondisk_log_size":8643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35377152,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6280,"num_write_kb":25120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4460}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8568","reported_seq":8689,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.699581+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:35.699581+0000","last_peered":"2024-09-11T01:17:35.699581+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:35.699581+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8568,"log_dups_size":0,"ondisk_log_size":8568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35086336,"num_objects":8566,"num_object_clones":0,"num_object_copies":17132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5258,"num_write_kb":21032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4402},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8738","reported_seq":8863,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.745242+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:35.745242+0000","last_peered":"2024-09-11T01:17:35.745242+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:35.745242+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8738,"log_dups_size":0,"ondisk_log_size":8738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4538},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8597","reported_seq":8716,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.701513+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:35.701513+0000","last_peered":"2024-09-11T01:17:35.701513+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:35.701513+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8597,"log_dups_size":0,"ondisk_log_size":8597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":20528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4445},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8464","reported_seq":8590,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.671432+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:35.671432+0000","last_peered":"2024-09-11T01:17:35.671432+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:35.671432+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8464,"log_dups_size":0,"ondisk_log_size":8464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34620358,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5121,"num_write_kb":20484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":559710942,"num_objects":136657,"num_object_clones":0,"num_object_copies":273314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136657,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":95792,"num_write_kb":383168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4126,"num_bytes_recovered":16892387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1139634176,"data_stored":1139482430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136729,"ondisk_log_size":136729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643292,"kb_used_data":404748,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93728548,"statfs":{"total":96636764160,"available":95978033152,"internally_reserved":0,"allocated":414461952,"data_stored":414205048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447908,"kb_used_data":231568,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93923932,"statfs":{"total":96636764160,"available":96178106368,"internally_reserved":0,"allocated":237125632,"data_stored":236816105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,0,0,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778156,"kb_used_data":475804,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93593684,"statfs":{"total":96636764160,"available":95839932416,"internally_reserved":0,"allocated":487223296,"data_stored":486962315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,3,9,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487763968,"data_stored":487727838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237510656,"data_stored":237431170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414359552,"data_stored":414323422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:38.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:39.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:39.166 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:39.188 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-11T01:17:38.471080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":559710961,"num_objects":136659,"num_object_clones":0,"num_object_copies":273318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136659,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":95794,"num_write_kb":383170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4127,"num_bytes_recovered":16892406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136732,"ondisk_log_size":136732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1869356,"kb_used_data":1112120,"kb_used_omap":12,"kb_used_meta":757171,"kb_avail":281246164,"statfs":{"total":289910292480,"available":287996071936,"internally_reserved":0,"allocated":1138810880,"data_stored":1137983468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":775343111},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,7,4,6,20,7,1,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-292,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3906,"num_write_kb":15624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003577"},"pg_stats":[{"pgid":"2.f","version":"39'8451","reported_seq":8515,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.698765+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:35.698765+0000","last_peered":"2024-09-11T01:17:35.698765+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:35.698765+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8451,"log_dups_size":0,"ondisk_log_size":8451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615296,"num_objects":8451,"num_object_clones":0,"num_object_copies":16902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8451,"num_write_kb":33804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4328},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"39'8549","reported_seq":8620,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.967804+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:34.967804+0000","last_peered":"2024-09-11T01:17:34.967804+0000","last_clean":"2024-09-11T01:17:34.967804+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:34.967804+0000","last_undegraded":"2024-09-11T01:17:34.967804+0000","last_fullsized":"2024-09-11T01:17:34.967804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8549,"log_dups_size":0,"ondisk_log_size":8549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34993094,"num_objects":8545,"num_object_clones":0,"num_object_copies":17090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8545,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6647,"num_write_kb":26588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"39'8393","reported_seq":8463,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.956861+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:34.956861+0000","last_peered":"2024-09-11T01:17:34.956861+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:34.956861+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8393,"log_dups_size":0,"ondisk_log_size":8393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34361344,"num_objects":8389,"num_object_clones":0,"num_object_copies":16778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"39'8469","reported_seq":8505,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.110790+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:35.110790+0000","last_peered":"2024-09-11T01:17:35.110790+0000","last_clean":"2024-09-11T01:17:35.110790+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:35.110790+0000","last_undegraded":"2024-09-11T01:17:35.110790+0000","last_fullsized":"2024-09-11T01:17:35.110790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8469,"num_write_kb":33876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8469","reported_seq":8581,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.714776+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:35.714776+0000","last_peered":"2024-09-11T01:17:35.714776+0000","last_clean":"2024-09-11T01:17:35.714776+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:35.714776+0000","last_undegraded":"2024-09-11T01:17:35.714776+0000","last_fullsized":"2024-09-11T01:17:35.714776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660835,"num_objects":8463,"num_object_clones":0,"num_object_copies":16926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5267,"num_write_kb":21068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8511","reported_seq":8636,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.685940+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:35.685940+0000","last_peered":"2024-09-11T01:17:35.685940+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:35.685940+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34841059,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4392},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8735","reported_seq":8858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.737508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:35.737508+0000","last_peered":"2024-09-11T01:17:35.737508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:35.737508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35754950,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5488,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8553","reported_seq":8663,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.650462+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:35.650462+0000","last_peered":"2024-09-11T01:17:35.650462+0000","last_clean":"2024-09-11T01:17:35.650462+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:35.650462+0000","last_undegraded":"2024-09-11T01:17:35.650462+0000","last_fullsized":"2024-09-11T01:17:35.650462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35024896,"num_objects":8551,"num_object_clones":0,"num_object_copies":17102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5506,"num_write_kb":22024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8587","reported_seq":16924,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:38.360037+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:38.360037+0000","last_peered":"2024-09-11T01:17:38.360037+0000","last_clean":"2024-09-11T01:17:38.360037+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:38.360037+0000","last_undegraded":"2024-09-11T01:17:38.360037+0000","last_fullsized":"2024-09-11T01:17:38.360037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.990410+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:15:19.990410+0000","last_peered":"2024-09-11T01:15:19.990410+0000","last_clean":"2024-09-11T01:15:19.990410+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:15:19.990410+0000","last_undegraded":"2024-09-11T01:15:19.990410+0000","last_fullsized":"2024-09-11T01:15:19.990410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8647","reported_seq":8766,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.645841+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:35.645841+0000","last_peered":"2024-09-11T01:17:35.645841+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:35.645841+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8647,"log_dups_size":0,"ondisk_log_size":8647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35385344,"num_objects":8639,"num_object_clones":0,"num_object_copies":17278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5011,"num_write_kb":20044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4534},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"39'8355","reported_seq":8440,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.091361+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:35.091361+0000","last_peered":"2024-09-11T01:17:35.091361+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:35.091361+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8355,"log_dups_size":0,"ondisk_log_size":8355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34202083,"num_objects":8351,"num_object_clones":0,"num_object_copies":16702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6124,"num_write_kb":24496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4324}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"39'8643","reported_seq":8727,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.071832+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:35.071832+0000","last_peered":"2024-09-11T01:17:35.071832+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:35.071832+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8643,"log_dups_size":0,"ondisk_log_size":8643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35377152,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6280,"num_write_kb":25120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4460}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8568","reported_seq":8689,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.699581+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:35.699581+0000","last_peered":"2024-09-11T01:17:35.699581+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:35.699581+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8568,"log_dups_size":0,"ondisk_log_size":8568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35086336,"num_objects":8566,"num_object_clones":0,"num_object_copies":17132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5258,"num_write_kb":21032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4402},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8738","reported_seq":8863,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.745242+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:35.745242+0000","last_peered":"2024-09-11T01:17:35.745242+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:35.745242+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8738,"log_dups_size":0,"ondisk_log_size":8738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4538},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.031492+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:15:20.031492+0000","last_peered":"2024-09-11T01:15:20.031492+0000","last_clean":"2024-09-11T01:15:20.031492+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:15:20.031492+0000","last_undegraded":"2024-09-11T01:15:20.031492+0000","last_fullsized":"2024-09-11T01:15:20.031492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8597","reported_seq":8716,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.701513+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:35.701513+0000","last_peered":"2024-09-11T01:17:35.701513+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:35.701513+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8597,"log_dups_size":0,"ondisk_log_size":8597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":20528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4445},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8464","reported_seq":8590,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.671432+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:35.671432+0000","last_peered":"2024-09-11T01:17:35.671432+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:35.671432+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8464,"log_dups_size":0,"ondisk_log_size":8464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34620358,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5121,"num_write_kb":20484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":559710942,"num_objects":136657,"num_object_clones":0,"num_object_copies":273314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":136657,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":95792,"num_write_kb":383168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4126,"num_bytes_recovered":16892387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1139634176,"data_stored":1139482430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136729,"ondisk_log_size":136729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643292,"kb_used_data":404748,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93728548,"statfs":{"total":96636764160,"available":95978033152,"internally_reserved":0,"allocated":414461952,"data_stored":414205048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":244248238},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447908,"kb_used_data":231568,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93923932,"statfs":{"total":96636764160,"available":96178106368,"internally_reserved":0,"allocated":237125632,"data_stored":236816105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,0,0,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778156,"kb_used_data":475804,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93593684,"statfs":{"total":96636764160,"available":95839932416,"internally_reserved":0,"allocated":487223296,"data_stored":486962315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,3,9,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487763968,"data_stored":487727838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237510656,"data_stored":237431170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414359552,"data_stored":414323422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "39'8451", 'reported_seq': 8515, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.698765+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:35.698765+0000', 'last_peered': '2024-09-11T01:17:35.698765+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:35.698765+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 8451, 'log_dups_size': 0, 'ondisk_log_size': 8451, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34615296, 'num_objects': 8451, 'num_object_clones': 0, 'num_object_copies': 16902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8451, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8451, 'num_write_kb': 33804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4328}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "39'8393", 'reported_seq': 8463, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:34.956861+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:34.956861+0000', 'last_peered': '2024-09-11T01:17:34.956861+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:34.956861+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 8393, 'log_dups_size': 0, 'ondisk_log_size': 8393, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34361344, 'num_objects': 8389, 'num_object_clones': 0, 'num_object_copies': 16778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6454, 'num_write_kb': 25816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 4384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "39'8511", 'reported_seq': 8636, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.685940+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:35.685940+0000', 'last_peered': '2024-09-11T01:17:35.685940+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:35.685940+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 8511, 'log_dups_size': 0, 'ondisk_log_size': 8511, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34841059, 'num_objects': 8507, 'num_object_clones': 0, 'num_object_copies': 17014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8507, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4999, 'num_write_kb': 19996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4392}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "39'8735", 'reported_seq': 8858, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.737508+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:35.737508+0000', 'last_peered': '2024-09-11T01:17:35.737508+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:35.737508+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 8735, 'log_dups_size': 0, 'ondisk_log_size': 8735, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35754950, 'num_objects': 8731, 'num_object_clones': 0, 'num_object_copies': 17462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8731, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5488, 'num_write_kb': 21952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4448}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "39'8647", 'reported_seq': 8766, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.645841+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:35.645841+0000', 'last_peered': '2024-09-11T01:17:35.645841+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:35.645841+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8647, 'log_dups_size': 0, 'ondisk_log_size': 8647, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35385344, 'num_objects': 8639, 'num_object_clones': 0, 'num_object_copies': 17278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5011, 'num_write_kb': 20044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4534}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "39'8355", 'reported_seq': 8440, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.091361+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:35.091361+0000', 'last_peered': '2024-09-11T01:17:35.091361+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:35.091361+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 8355, 'log_dups_size': 0, 'ondisk_log_size': 8355, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34202083, 'num_objects': 8351, 'num_object_clones': 0, 'num_object_copies': 16702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8351, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6124, 'num_write_kb': 24496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 4324}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "39'8643", 'reported_seq': 8727, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.071832+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:35.071832+0000', 'last_peered': '2024-09-11T01:17:35.071832+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:35.071832+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8643, 'log_dups_size': 0, 'ondisk_log_size': 8643, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35377152, 'num_objects': 8637, 'num_object_clones': 0, 'num_object_copies': 17274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6280, 'num_write_kb': 25120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4460}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "39'8568", 'reported_seq': 8689, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.699581+0000', 'last_change': '2024-09-11T01:15:20.481087+0000', 'last_active': '2024-09-11T01:17:35.699581+0000', 'last_peered': '2024-09-11T01:17:35.699581+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:35.699581+0000', 'last_undegraded': '2024-09-11T01:15:20.082638+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 8568, 'log_dups_size': 0, 'ondisk_log_size': 8568, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35086336, 'num_objects': 8566, 'num_object_clones': 0, 'num_object_copies': 17132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5258, 'num_write_kb': 21032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4402}, {'shards': '0,2', 'objects': 4164}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "39'8738", 'reported_seq': 8863, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.745242+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:35.745242+0000', 'last_peered': '2024-09-11T01:17:35.745242+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:35.745242+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 8738, 'log_dups_size': 0, 'ondisk_log_size': 8738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35758080, 'num_objects': 8730, 'num_object_clones': 0, 'num_object_copies': 17460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5131, 'num_write_kb': 20524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4538}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:39.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "39'8597", 'reported_seq': 8716, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.701513+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:35.701513+0000', 'last_peered': '2024-09-11T01:17:35.701513+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:35.701513+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 8597, 'log_dups_size': 0, 'ondisk_log_size': 8597, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35196928, 'num_objects': 8593, 'num_object_clones': 0, 'num_object_copies': 17186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5132, 'num_write_kb': 20528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4445}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:39.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "39'8464", 'reported_seq': 8590, 'reported_epoch': 39, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:35.671432+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:35.671432+0000', 'last_peered': '2024-09-11T01:17:35.671432+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:35.671432+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 8464, 'log_dups_size': 0, 'ondisk_log_size': 8464, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34620358, 'num_objects': 8454, 'num_object_clones': 0, 'num_object_copies': 16908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8454, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5121, 'num_write_kb': 20484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4300}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.192 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:39.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 183 20 138153 138133 2.9479 1.67578 0.124036 0.0264939 2024-09-11T01:17:39.911 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:39.911 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45508/273318 objects degraded (16.650%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11}],"num_pgs":24,"num_pools":2,"num_objects":136659,"data_bytes":559710961,"bytes_used":1914220544,"bytes_avail":287996071936,"bytes_total":289910292480,"degraded_objects":45508,"degraded_total":273318,"degraded_ratio":0.16650202328423303,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":149437,"recovering_keys_per_sec":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"write_bytes_sec":1998978,"read_op_per_sec":0,"write_op_per_sec":488},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:39.935 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:39.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:40.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 184 20 138583 138563 2.94101 1.67969 0.0429002 0.0265553 2024-09-11T01:17:40.509 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:40.512 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:40.535 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2024-09-11T01:17:40.471537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":562451185,"num_objects":137328,"num_object_clones":0,"num_object_copies":274656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137328,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":96463,"num_write_kb":385846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4127,"num_bytes_recovered":16892406,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137401,"ondisk_log_size":137401,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1879180,"kb_used_data":1116376,"kb_used_omap":12,"kb_used_meta":762739,"kb_avail":281236340,"statfs":{"total":289910292480,"available":287986012160,"internally_reserved":0,"allocated":1143169024,"data_stored":1142331211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":781044743},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,2,5,13,4,1,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12578816,"num_objects":3071,"num_object_clones":0,"num_object_copies":6142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-292,"num_objects_unfound":0,"num_objects_dirty":3071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3071,"num_write_kb":12284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003460"},"pg_stats":[{"pgid":"2.f","version":"39'8451","reported_seq":8515,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.698765+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:35.698765+0000","last_peered":"2024-09-11T01:17:35.698765+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:35.698765+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8451,"log_dups_size":0,"ondisk_log_size":8451,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615296,"num_objects":8451,"num_object_clones":0,"num_object_copies":16902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8451,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8451,"num_write_kb":33804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4328},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8678","reported_seq":8753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.048217+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:40.048217+0000","last_peered":"2024-09-11T01:17:40.048217+0000","last_clean":"2024-09-11T01:17:40.048217+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:40.048217+0000","last_undegraded":"2024-09-11T01:17:40.048217+0000","last_fullsized":"2024-09-11T01:17:40.048217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35521478,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6776,"num_write_kb":27104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8518","reported_seq":8596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.070451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:40.070451+0000","last_peered":"2024-09-11T01:17:40.070451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:40.070451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8518,"log_dups_size":0,"ondisk_log_size":8518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8618","reported_seq":8658,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.134752+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:40.134752+0000","last_peered":"2024-09-11T01:17:40.134752+0000","last_clean":"2024-09-11T01:17:40.134752+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:40.134752+0000","last_undegraded":"2024-09-11T01:17:40.134752+0000","last_fullsized":"2024-09-11T01:17:40.134752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8618,"log_dups_size":0,"ondisk_log_size":8618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":34472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"39'8469","reported_seq":8581,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.714776+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:35.714776+0000","last_peered":"2024-09-11T01:17:35.714776+0000","last_clean":"2024-09-11T01:17:35.714776+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:35.714776+0000","last_undegraded":"2024-09-11T01:17:35.714776+0000","last_fullsized":"2024-09-11T01:17:35.714776+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8469,"log_dups_size":0,"ondisk_log_size":8469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660835,"num_objects":8463,"num_object_clones":0,"num_object_copies":16926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5267,"num_write_kb":21068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"39'8511","reported_seq":8636,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.685940+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:35.685940+0000","last_peered":"2024-09-11T01:17:35.685940+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:35.685940+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8511,"log_dups_size":0,"ondisk_log_size":8511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34841059,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4392},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"39'8735","reported_seq":8858,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.737508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:35.737508+0000","last_peered":"2024-09-11T01:17:35.737508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:35.737508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35754950,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5488,"num_write_kb":21952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4448},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"39'8553","reported_seq":8663,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:17:35.650462+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:35.650462+0000","last_peered":"2024-09-11T01:17:35.650462+0000","last_clean":"2024-09-11T01:17:35.650462+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:35.650462+0000","last_undegraded":"2024-09-11T01:17:35.650462+0000","last_fullsized":"2024-09-11T01:17:35.650462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8553,"log_dups_size":0,"ondisk_log_size":8553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35024896,"num_objects":8551,"num_object_clones":0,"num_object_copies":17102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8551,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5506,"num_write_kb":22024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8587","reported_seq":16924,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:38.360037+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:38.360037+0000","last_peered":"2024-09-11T01:17:38.360037+0000","last_clean":"2024-09-11T01:17:38.360037+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:38.360037+0000","last_undegraded":"2024-09-11T01:17:38.360037+0000","last_fullsized":"2024-09-11T01:17:38.360037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"39'8647","reported_seq":8766,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.645841+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:35.645841+0000","last_peered":"2024-09-11T01:17:35.645841+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:35.645841+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8647,"log_dups_size":0,"ondisk_log_size":8647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35385344,"num_objects":8639,"num_object_clones":0,"num_object_copies":17278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5011,"num_write_kb":20044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4534},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":88,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-11T01:15:20.104996+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:15:20.104996+0000","last_peered":"2024-09-11T01:15:20.104996+0000","last_clean":"2024-09-11T01:15:20.104996+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:15:20.104996+0000","last_undegraded":"2024-09-11T01:15:20.104996+0000","last_fullsized":"2024-09-11T01:15:20.104996+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8496","reported_seq":8585,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:39.978509+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:39.978509+0000","last_peered":"2024-09-11T01:17:39.978509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:39.978509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779619,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6265,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4465}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8768","reported_seq":8856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.149993+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:40.149993+0000","last_peered":"2024-09-11T01:17:40.149993+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:40.149993+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35889152,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6405,"num_write_kb":25620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"39'8568","reported_seq":8689,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.699581+0000","last_change":"2024-09-11T01:15:20.481087+0000","last_active":"2024-09-11T01:17:35.699581+0000","last_peered":"2024-09-11T01:17:35.699581+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:35.699581+0000","last_undegraded":"2024-09-11T01:15:20.082638+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8568,"log_dups_size":0,"ondisk_log_size":8568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35086336,"num_objects":8566,"num_object_clones":0,"num_object_copies":17132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5258,"num_write_kb":21032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4402},{"shards":"0,2","objects":4164}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"39'8738","reported_seq":8863,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.745242+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:35.745242+0000","last_peered":"2024-09-11T01:17:35.745242+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:35.745242+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8738,"log_dups_size":0,"ondisk_log_size":8738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758080,"num_objects":8730,"num_object_clones":0,"num_object_copies":17460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4538},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"39'8597","reported_seq":8716,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.701513+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:35.701513+0000","last_peered":"2024-09-11T01:17:35.701513+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:35.701513+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8597,"log_dups_size":0,"ondisk_log_size":8597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5132,"num_write_kb":20528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4445},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"39'8464","reported_seq":8590,"reported_epoch":39,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:35.671432+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:35.671432+0000","last_peered":"2024-09-11T01:17:35.671432+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:35.671432+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8464,"log_dups_size":0,"ondisk_log_size":8464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34620358,"num_objects":8454,"num_object_clones":0,"num_object_copies":16908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8454,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5121,"num_write_kb":20484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":562451166,"num_objects":137326,"num_object_clones":0,"num_object_copies":274652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137326,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":96461,"num_write_kb":385844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4126,"num_bytes_recovered":16892387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1143959552,"data_stored":1143807806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137398,"ondisk_log_size":137398,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653116,"kb_used_data":409004,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93718724,"statfs":{"total":96636764160,"available":95967973376,"internally_reserved":0,"allocated":418820096,"data_stored":418552791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447908,"kb_used_data":231568,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93923932,"statfs":{"total":96636764160,"available":96178106368,"internally_reserved":0,"allocated":237125632,"data_stored":236816105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,0,0,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":778156,"kb_used_data":475804,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93593684,"statfs":{"total":96636764160,"available":95839932416,"internally_reserved":0,"allocated":487223296,"data_stored":486962315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,3,9,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":487763968,"data_stored":487727838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237510656,"data_stored":237431170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418684928,"data_stored":418648798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:41.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 185 20 139064 139044 2.93527 1.87891 0.0836495 0.0266093 2024-09-11T01:17:42.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 186 20 139553 139533 2.92975 1.91016 0.022508 0.026659 2024-09-11T01:17:43.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 187 20 140025 140005 2.92395 1.84375 0.0879185 0.0267109 2024-09-11T01:17:43.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:44.123 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:44.125 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:44.151 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-11T01:17:42.471982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":567796465,"num_objects":138633,"num_object_clones":0,"num_object_copies":277266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":138633,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":97768,"num_write_kb":391066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4220,"num_bytes_recovered":17273334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138706,"ondisk_log_size":138706,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1889640,"kb_used_data":1122676,"kb_used_omap":12,"kb_used_meta":766899,"kb_avail":281225880,"statfs":{"total":289910292480,"available":287975301120,"internally_reserved":0,"allocated":1149620224,"data_stored":1148772010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":785304583},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,2,3,8,5,1,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17924096,"num_objects":4376,"num_object_clones":0,"num_object_copies":8752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4165,"num_objects_misplaced":3779,"num_objects_unfound":0,"num_objects_dirty":4376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4376,"num_write_kb":17504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003306"},"pg_stats":[{"pgid":"2.f","version":"41'8576","reported_seq":8642,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.750896+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:40.750896+0000","last_peered":"2024-09-11T01:17:40.750896+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:40.750896+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8576,"log_dups_size":0,"ondisk_log_size":8576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8576,"num_write_kb":34304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8678","reported_seq":8753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.048217+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:40.048217+0000","last_peered":"2024-09-11T01:17:40.048217+0000","last_clean":"2024-09-11T01:17:40.048217+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:40.048217+0000","last_undegraded":"2024-09-11T01:17:40.048217+0000","last_fullsized":"2024-09-11T01:17:40.048217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35521478,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6776,"num_write_kb":27104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8518","reported_seq":8596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.070451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:40.070451+0000","last_peered":"2024-09-11T01:17:40.070451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:40.070451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8518,"log_dups_size":0,"ondisk_log_size":8518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8618","reported_seq":8658,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.134752+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:40.134752+0000","last_peered":"2024-09-11T01:17:40.134752+0000","last_clean":"2024-09-11T01:17:40.134752+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:40.134752+0000","last_undegraded":"2024-09-11T01:17:40.134752+0000","last_fullsized":"2024-09-11T01:17:40.134752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8618,"log_dups_size":0,"ondisk_log_size":8618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":34472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8606","reported_seq":8722,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.742119+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:40.742119+0000","last_peered":"2024-09-11T01:17:40.742119+0000","last_clean":"2024-09-11T01:17:40.742119+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:40.742119+0000","last_undegraded":"2024-09-11T01:17:40.742119+0000","last_fullsized":"2024-09-11T01:17:40.742119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8606,"log_dups_size":0,"ondisk_log_size":8606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35221987,"num_objects":8600,"num_object_clones":0,"num_object_copies":17200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5404,"num_write_kb":21616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8641","reported_seq":8770,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.762106+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:40.762106+0000","last_peered":"2024-09-11T01:17:40.762106+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:40.762106+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8641,"log_dups_size":0,"ondisk_log_size":8641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35373539,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8860","reported_seq":8981,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.694792+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:40.694792+0000","last_peered":"2024-09-11T01:17:40.694792+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:40.694792+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8860,"log_dups_size":0,"ondisk_log_size":8860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36266950,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5613,"num_write_kb":22452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8682","reported_seq":8796,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.664216+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:40.664216+0000","last_peered":"2024-09-11T01:17:40.664216+0000","last_clean":"2024-09-11T01:17:40.664216+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:40.664216+0000","last_undegraded":"2024-09-11T01:17:40.664216+0000","last_fullsized":"2024-09-11T01:17:40.664216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8682,"log_dups_size":0,"ondisk_log_size":8682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":22540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8587","reported_seq":16924,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:38.360037+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:38.360037+0000","last_peered":"2024-09-11T01:17:38.360037+0000","last_clean":"2024-09-11T01:17:38.360037+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:38.360037+0000","last_undegraded":"2024-09-11T01:17:38.360037+0000","last_fullsized":"2024-09-11T01:17:38.360037+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8587,"log_dups_size":0,"ondisk_log_size":8587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35144163,"num_objects":8581,"num_object_clones":0,"num_object_copies":17162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8771","reported_seq":8895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.727851+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:40.727851+0000","last_peered":"2024-09-11T01:17:40.727851+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:40.727851+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8771,"log_dups_size":0,"ondisk_log_size":8771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35893248,"num_objects":8763,"num_object_clones":0,"num_object_copies":17526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5135,"num_write_kb":20540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4658},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8496","reported_seq":8585,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:39.978509+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:39.978509+0000","last_peered":"2024-09-11T01:17:39.978509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:39.978509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779619,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6265,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4465}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8768","reported_seq":8856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.149993+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:40.149993+0000","last_peered":"2024-09-11T01:17:40.149993+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:40.149993+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35889152,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6405,"num_write_kb":25620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8699","reported_seq":9011,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:40.763561+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:40.763561+0000","last_peered":"2024-09-11T01:17:40.763561+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:40.763561+0000","last_undegraded":"2024-09-11T01:17:40.763561+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":380928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4626},{"shards":"0,2","objects":4071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8870","reported_seq":8999,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.730599+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:40.730599+0000","last_peered":"2024-09-11T01:17:40.730599+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:40.730599+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5263,"num_write_kb":21052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4670},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8737","reported_seq":8864,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.742303+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:40.742303+0000","last_peered":"2024-09-11T01:17:40.742303+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:40.742303+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8737,"log_dups_size":0,"ondisk_log_size":8737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35770368,"num_objects":8733,"num_object_clones":0,"num_object_copies":17466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":21088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4585},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8596","reported_seq":8726,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.752317+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:40.752317+0000","last_peered":"2024-09-11T01:17:40.752317+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:40.752317+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8596,"log_dups_size":0,"ondisk_log_size":8596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35161030,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5253,"num_write_kb":21012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4432},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":567796446,"num_objects":138631,"num_object_clones":0,"num_object_copies":277262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":138631,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":97766,"num_write_kb":391064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4219,"num_bytes_recovered":17273315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1150406656,"data_stored":1150254910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138703,"ondisk_log_size":138703,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653116,"kb_used_data":409004,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93718724,"statfs":{"total":96636764160,"available":95967973376,"internally_reserved":0,"allocated":418820096,"data_stored":418552791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789981,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447908,"kb_used_data":231568,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93923932,"statfs":{"total":96636764160,"available":96178106368,"internally_reserved":0,"allocated":237125632,"data_stored":236816105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,0,0,5,4],"upper_bound":1024},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788616,"kb_used_data":482104,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93583224,"statfs":{"total":96636764160,"available":95829221376,"internally_reserved":0,"allocated":493674496,"data_stored":493403114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":494211072,"data_stored":494174942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":237510656,"data_stored":237431170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418684928,"data_stored":418648798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:44.152 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:44.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 188 20 140468 140448 2.9176 1.73047 0.0299028 0.0267685 2024-09-11T01:17:44.713 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:44.714 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:44.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-11T01:17:44.472465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568304369,"num_objects":138757,"num_object_clones":0,"num_object_copies":277514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":138757,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":97892,"num_write_kb":391562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4220,"num_bytes_recovered":17273334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138830,"ondisk_log_size":138830,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900688,"kb_used_data":1129568,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281214832,"statfs":{"total":289910292480,"available":287963987968,"internally_reserved":0,"allocated":1156677632,"data_stored":1155824207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":789564423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,5,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8925184,"num_objects":2179,"num_object_clones":0,"num_object_copies":4358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4165,"num_objects_misplaced":4004,"num_objects_unfound":0,"num_objects_dirty":2179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":8716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003317"},"pg_stats":[{"pgid":"2.f","version":"41'8576","reported_seq":8642,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.750896+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:40.750896+0000","last_peered":"2024-09-11T01:17:40.750896+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:40.750896+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8576,"log_dups_size":0,"ondisk_log_size":8576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8576,"num_write_kb":34304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8678","reported_seq":8753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.048217+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:40.048217+0000","last_peered":"2024-09-11T01:17:40.048217+0000","last_clean":"2024-09-11T01:17:40.048217+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:40.048217+0000","last_undegraded":"2024-09-11T01:17:40.048217+0000","last_fullsized":"2024-09-11T01:17:40.048217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35521478,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6776,"num_write_kb":27104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8518","reported_seq":8596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.070451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:40.070451+0000","last_peered":"2024-09-11T01:17:40.070451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:40.070451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8518,"log_dups_size":0,"ondisk_log_size":8518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8618","reported_seq":8658,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.134752+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:40.134752+0000","last_peered":"2024-09-11T01:17:40.134752+0000","last_clean":"2024-09-11T01:17:40.134752+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:40.134752+0000","last_undegraded":"2024-09-11T01:17:40.134752+0000","last_fullsized":"2024-09-11T01:17:40.134752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8618,"log_dups_size":0,"ondisk_log_size":8618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":34472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8606","reported_seq":8722,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.742119+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:40.742119+0000","last_peered":"2024-09-11T01:17:40.742119+0000","last_clean":"2024-09-11T01:17:40.742119+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:40.742119+0000","last_undegraded":"2024-09-11T01:17:40.742119+0000","last_fullsized":"2024-09-11T01:17:40.742119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8606,"log_dups_size":0,"ondisk_log_size":8606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35221987,"num_objects":8600,"num_object_clones":0,"num_object_copies":17200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5404,"num_write_kb":21616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8641","reported_seq":8770,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.762106+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:40.762106+0000","last_peered":"2024-09-11T01:17:40.762106+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:40.762106+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8641,"log_dups_size":0,"ondisk_log_size":8641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35373539,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8860","reported_seq":8981,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.694792+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:40.694792+0000","last_peered":"2024-09-11T01:17:40.694792+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:40.694792+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8860,"log_dups_size":0,"ondisk_log_size":8860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36266950,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5613,"num_write_kb":22452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8682","reported_seq":8796,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.664216+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:40.664216+0000","last_peered":"2024-09-11T01:17:40.664216+0000","last_clean":"2024-09-11T01:17:40.664216+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:40.664216+0000","last_undegraded":"2024-09-11T01:17:40.664216+0000","last_fullsized":"2024-09-11T01:17:40.664216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8682,"log_dups_size":0,"ondisk_log_size":8682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":22540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8711","reported_seq":17049,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:43.359173+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:43.359173+0000","last_peered":"2024-09-11T01:17:43.359173+0000","last_clean":"2024-09-11T01:17:43.359173+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:43.359173+0000","last_undegraded":"2024-09-11T01:17:43.359173+0000","last_fullsized":"2024-09-11T01:17:43.359173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8711,"log_dups_size":0,"ondisk_log_size":8711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35652067,"num_objects":8705,"num_object_clones":0,"num_object_copies":17410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8705,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6578,"num_write_kb":26312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8771","reported_seq":8895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.727851+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:40.727851+0000","last_peered":"2024-09-11T01:17:40.727851+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:40.727851+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8771,"log_dups_size":0,"ondisk_log_size":8771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35893248,"num_objects":8763,"num_object_clones":0,"num_object_copies":17526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5135,"num_write_kb":20540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4658},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8496","reported_seq":8585,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:39.978509+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:39.978509+0000","last_peered":"2024-09-11T01:17:39.978509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:39.978509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779619,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6265,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4465}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8768","reported_seq":8856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.149993+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:40.149993+0000","last_peered":"2024-09-11T01:17:40.149993+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:40.149993+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35889152,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6405,"num_write_kb":25620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8699","reported_seq":9011,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:40.763561+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:40.763561+0000","last_peered":"2024-09-11T01:17:40.763561+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:40.763561+0000","last_undegraded":"2024-09-11T01:17:40.763561+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":380928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4626},{"shards":"0,2","objects":4071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8870","reported_seq":8999,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.730599+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:40.730599+0000","last_peered":"2024-09-11T01:17:40.730599+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:40.730599+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5263,"num_write_kb":21052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4670},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8737","reported_seq":8864,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.742303+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:40.742303+0000","last_peered":"2024-09-11T01:17:40.742303+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:40.742303+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8737,"log_dups_size":0,"ondisk_log_size":8737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35770368,"num_objects":8733,"num_object_clones":0,"num_object_copies":17466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":21088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4585},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8596","reported_seq":8726,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.752317+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:40.752317+0000","last_peered":"2024-09-11T01:17:40.752317+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:40.752317+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8596,"log_dups_size":0,"ondisk_log_size":8596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35161030,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5253,"num_write_kb":21012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4432},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568304350,"num_objects":138755,"num_object_clones":0,"num_object_copies":277510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":138755,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":97890,"num_write_kb":391560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4219,"num_bytes_recovered":17273315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1157783552,"data_stored":1157631806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138827,"ondisk_log_size":138827,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653116,"kb_used_data":409004,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93718724,"statfs":{"total":96636764160,"available":95967973376,"internally_reserved":0,"allocated":418820096,"data_stored":418552791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789982,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458956,"kb_used_data":238460,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93912884,"statfs":{"total":96636764160,"available":96166793216,"internally_reserved":0,"allocated":244183040,"data_stored":243868302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788616,"kb_used_data":482104,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93583224,"statfs":{"total":96636764160,"available":95829221376,"internally_reserved":0,"allocated":493674496,"data_stored":493403114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":494211072,"data_stored":494174942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244887552,"data_stored":244808066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418684928,"data_stored":418648798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'8576", 'reported_seq': 8642, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.750896+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:40.750896+0000', 'last_peered': '2024-09-11T01:17:40.750896+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:40.750896+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 8576, 'log_dups_size': 0, 'ondisk_log_size': 8576, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35127296, 'num_objects': 8576, 'num_object_clones': 0, 'num_object_copies': 17152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8576, 'num_write_kb': 34304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4453}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'8518", 'reported_seq': 8596, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.070451+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:40.070451+0000', 'last_peered': '2024-09-11T01:17:40.070451+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:40.070451+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 8518, 'log_dups_size': 0, 'ondisk_log_size': 8518, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34873344, 'num_objects': 8514, 'num_object_clones': 0, 'num_object_copies': 17028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8514, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6579, 'num_write_kb': 26316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 4509}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'8641", 'reported_seq': 8770, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.762106+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:40.762106+0000', 'last_peered': '2024-09-11T01:17:40.762106+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:40.762106+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 8641, 'log_dups_size': 0, 'ondisk_log_size': 8641, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35373539, 'num_objects': 8637, 'num_object_clones': 0, 'num_object_copies': 17274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8637, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5129, 'num_write_kb': 20516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4522}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'8860", 'reported_seq': 8981, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.694792+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:40.694792+0000', 'last_peered': '2024-09-11T01:17:40.694792+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:40.694792+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 8860, 'log_dups_size': 0, 'ondisk_log_size': 8860, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36266950, 'num_objects': 8856, 'num_object_clones': 0, 'num_object_copies': 17712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8856, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5613, 'num_write_kb': 22452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4573}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:44.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'8771", 'reported_seq': 8895, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.727851+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:40.727851+0000', 'last_peered': '2024-09-11T01:17:40.727851+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:40.727851+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8771, 'log_dups_size': 0, 'ondisk_log_size': 8771, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35893248, 'num_objects': 8763, 'num_object_clones': 0, 'num_object_copies': 17526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8763, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5135, 'num_write_kb': 20540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4658}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'8496", 'reported_seq': 8585, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:39.978509+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:39.978509+0000', 'last_peered': '2024-09-11T01:17:39.978509+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:39.978509+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 8496, 'log_dups_size': 0, 'ondisk_log_size': 8496, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34779619, 'num_objects': 8492, 'num_object_clones': 0, 'num_object_copies': 16984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8492, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6265, 'num_write_kb': 25060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 4465}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'8768", 'reported_seq': 8856, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.149993+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:40.149993+0000', 'last_peered': '2024-09-11T01:17:40.149993+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:40.149993+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8768, 'log_dups_size': 0, 'ondisk_log_size': 8768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35889152, 'num_objects': 8762, 'num_object_clones': 0, 'num_object_copies': 17524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6405, 'num_write_kb': 25620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4585}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'8699", 'reported_seq': 9011, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:40.763561+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:17:40.763561+0000', 'last_peered': '2024-09-11T01:17:40.763561+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:40.763561+0000', 'last_undegraded': '2024-09-11T01:17:40.763561+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 8699, 'log_dups_size': 0, 'ondisk_log_size': 8699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35622912, 'num_objects': 8697, 'num_object_clones': 0, 'num_object_copies': 17394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4071, 'num_objects_unfound': 0, 'num_objects_dirty': 8697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5389, 'num_write_kb': 21556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 93, 'num_bytes_recovered': 380928, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4626}, {'shards': '0,2', 'objects': 4071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'8870", 'reported_seq': 8999, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.730599+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:40.730599+0000', 'last_peered': '2024-09-11T01:17:40.730599+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:40.730599+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 8870, 'log_dups_size': 0, 'ondisk_log_size': 8870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36298752, 'num_objects': 8862, 'num_object_clones': 0, 'num_object_copies': 17724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5263, 'num_write_kb': 21052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4670}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:44.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'8737", 'reported_seq': 8864, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.742303+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:40.742303+0000', 'last_peered': '2024-09-11T01:17:40.742303+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:40.742303+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 8737, 'log_dups_size': 0, 'ondisk_log_size': 8737, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35770368, 'num_objects': 8733, 'num_object_clones': 0, 'num_object_copies': 17466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8733, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5272, 'num_write_kb': 21088, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4585}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:44.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'8596", 'reported_seq': 8726, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:40.752317+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:40.752317+0000', 'last_peered': '2024-09-11T01:17:40.752317+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:40.752317+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 8596, 'log_dups_size': 0, 'ondisk_log_size': 8596, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35161030, 'num_objects': 8586, 'num_object_clones': 0, 'num_object_copies': 17172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8586, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5253, 'num_write_kb': 21012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4432}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:44.744 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:45.351 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 189 20 140892 140872 2.91092 1.65625 0.0122522 0.0268299 2024-09-11T01:17:45.463 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:45.464 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/277514 objects degraded (14.898%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":265,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138757,"data_bytes":568304369,"bytes_used":1946304512,"bytes_avail":287963987968,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":277514,"degraded_ratio":0.14897626786396362,"misplaced_objects":4071,"misplaced_total":277514,"misplaced_ratio":0.014669530185864497,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":81886,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"write_bytes_sec":1115185,"read_op_per_sec":0,"write_op_per_sec":272},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:45.489 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:45.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:46.109 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:46.109 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:46.141 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-11T01:17:44.472465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568304369,"num_objects":138757,"num_object_clones":0,"num_object_copies":277514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":138757,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":97892,"num_write_kb":391562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4220,"num_bytes_recovered":17273334,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138830,"ondisk_log_size":138830,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900688,"kb_used_data":1129568,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281214832,"statfs":{"total":289910292480,"available":287963987968,"internally_reserved":0,"allocated":1156677632,"data_stored":1155824207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":789564423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,5,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8925184,"num_objects":2179,"num_object_clones":0,"num_object_copies":4358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4165,"num_objects_misplaced":4004,"num_objects_unfound":0,"num_objects_dirty":2179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2179,"num_write_kb":8716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003317"},"pg_stats":[{"pgid":"2.f","version":"41'8576","reported_seq":8642,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.750896+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:40.750896+0000","last_peered":"2024-09-11T01:17:40.750896+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:40.750896+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8576,"log_dups_size":0,"ondisk_log_size":8576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8576,"num_write_kb":34304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4453},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8678","reported_seq":8753,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.048217+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:40.048217+0000","last_peered":"2024-09-11T01:17:40.048217+0000","last_clean":"2024-09-11T01:17:40.048217+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:40.048217+0000","last_undegraded":"2024-09-11T01:17:40.048217+0000","last_fullsized":"2024-09-11T01:17:40.048217+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8678,"log_dups_size":0,"ondisk_log_size":8678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35521478,"num_objects":8674,"num_object_clones":0,"num_object_copies":17348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8674,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6776,"num_write_kb":27104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8518","reported_seq":8596,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.070451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:40.070451+0000","last_peered":"2024-09-11T01:17:40.070451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:40.070451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8518,"log_dups_size":0,"ondisk_log_size":8518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34873344,"num_objects":8514,"num_object_clones":0,"num_object_copies":17028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6579,"num_write_kb":26316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8618","reported_seq":8658,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.134752+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:40.134752+0000","last_peered":"2024-09-11T01:17:40.134752+0000","last_clean":"2024-09-11T01:17:40.134752+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:40.134752+0000","last_undegraded":"2024-09-11T01:17:40.134752+0000","last_fullsized":"2024-09-11T01:17:40.134752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8618,"log_dups_size":0,"ondisk_log_size":8618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35299328,"num_objects":8618,"num_object_clones":0,"num_object_copies":17236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8618,"num_write_kb":34472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8606","reported_seq":8722,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.742119+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:40.742119+0000","last_peered":"2024-09-11T01:17:40.742119+0000","last_clean":"2024-09-11T01:17:40.742119+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:40.742119+0000","last_undegraded":"2024-09-11T01:17:40.742119+0000","last_fullsized":"2024-09-11T01:17:40.742119+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8606,"log_dups_size":0,"ondisk_log_size":8606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35221987,"num_objects":8600,"num_object_clones":0,"num_object_copies":17200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5404,"num_write_kb":21616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8641","reported_seq":8770,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.762106+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:40.762106+0000","last_peered":"2024-09-11T01:17:40.762106+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:40.762106+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8641,"log_dups_size":0,"ondisk_log_size":8641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35373539,"num_objects":8637,"num_object_clones":0,"num_object_copies":17274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'8860","reported_seq":8981,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.694792+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:40.694792+0000","last_peered":"2024-09-11T01:17:40.694792+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:40.694792+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":8860,"log_dups_size":0,"ondisk_log_size":8860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36266950,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5613,"num_write_kb":22452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8682","reported_seq":8796,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:40.664216+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:40.664216+0000","last_peered":"2024-09-11T01:17:40.664216+0000","last_clean":"2024-09-11T01:17:40.664216+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:40.664216+0000","last_undegraded":"2024-09-11T01:17:40.664216+0000","last_fullsized":"2024-09-11T01:17:40.664216+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8682,"log_dups_size":0,"ondisk_log_size":8682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":22540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8711","reported_seq":17049,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:43.359173+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:43.359173+0000","last_peered":"2024-09-11T01:17:43.359173+0000","last_clean":"2024-09-11T01:17:43.359173+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:43.359173+0000","last_undegraded":"2024-09-11T01:17:43.359173+0000","last_fullsized":"2024-09-11T01:17:43.359173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8711,"log_dups_size":0,"ondisk_log_size":8711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35652067,"num_objects":8705,"num_object_clones":0,"num_object_copies":17410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8705,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6578,"num_write_kb":26312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8771","reported_seq":8895,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.727851+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:40.727851+0000","last_peered":"2024-09-11T01:17:40.727851+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:40.727851+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8771,"log_dups_size":0,"ondisk_log_size":8771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35893248,"num_objects":8763,"num_object_clones":0,"num_object_copies":17526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5135,"num_write_kb":20540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4658},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8496","reported_seq":8585,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:39.978509+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:39.978509+0000","last_peered":"2024-09-11T01:17:39.978509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:39.978509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779619,"num_objects":8492,"num_object_clones":0,"num_object_copies":16984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8492,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6265,"num_write_kb":25060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4465}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8768","reported_seq":8856,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.149993+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:40.149993+0000","last_peered":"2024-09-11T01:17:40.149993+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:40.149993+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35889152,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6405,"num_write_kb":25620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4585}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8699","reported_seq":9011,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:40.763561+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:40.763561+0000","last_peered":"2024-09-11T01:17:40.763561+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:40.763561+0000","last_undegraded":"2024-09-11T01:17:40.763561+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":21556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":380928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4626},{"shards":"0,2","objects":4071}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'8870","reported_seq":8999,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.730599+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:40.730599+0000","last_peered":"2024-09-11T01:17:40.730599+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:40.730599+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5263,"num_write_kb":21052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4670},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8737","reported_seq":8864,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.742303+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:40.742303+0000","last_peered":"2024-09-11T01:17:40.742303+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:40.742303+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8737,"log_dups_size":0,"ondisk_log_size":8737,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35770368,"num_objects":8733,"num_object_clones":0,"num_object_copies":17466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5272,"num_write_kb":21088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4585},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8596","reported_seq":8726,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:40.752317+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:40.752317+0000","last_peered":"2024-09-11T01:17:40.752317+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:40.752317+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8596,"log_dups_size":0,"ondisk_log_size":8596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35161030,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5253,"num_write_kb":21012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4432},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568304350,"num_objects":138755,"num_object_clones":0,"num_object_copies":277510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4071,"num_objects_unfound":0,"num_objects_dirty":138755,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":97890,"num_write_kb":391560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4219,"num_bytes_recovered":17273315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1157783552,"data_stored":1157631806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138827,"ondisk_log_size":138827,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653116,"kb_used_data":409004,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93718724,"statfs":{"total":96636764160,"available":95967973376,"internally_reserved":0,"allocated":418820096,"data_stored":418552791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789982,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458956,"kb_used_data":238460,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93912884,"statfs":{"total":96636764160,"available":96166793216,"internally_reserved":0,"allocated":244183040,"data_stored":243868302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":788616,"kb_used_data":482104,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93583224,"statfs":{"total":96636764160,"available":95829221376,"internally_reserved":0,"allocated":493674496,"data_stored":493403114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":494211072,"data_stored":494174942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":244887552,"data_stored":244808066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418684928,"data_stored":418648798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:46.352 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 190 20 141307 141287 2.90414 1.62109 0.0255562 0.0268933 2024-09-11T01:17:47.352 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 191 20 141800 141780 2.89901 1.92578 0.0254577 0.0269417 2024-09-11T01:17:48.354 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 192 20 142283 142263 2.89374 1.88672 0.0446301 0.0269919 2024-09-11T01:17:49.143 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:49.352 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 193 20 142754 142734 2.88828 1.83984 0.0224447 0.0270405 2024-09-11T01:17:49.722 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:49.722 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:49.752 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-11T01:17:48.477018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577798897,"num_objects":141075,"num_object_clones":0,"num_object_copies":282150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":141075,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":100210,"num_write_kb":400834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4281,"num_bytes_recovered":17523190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141148,"ondisk_log_size":141148,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1919004,"kb_used_data":1147884,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281196516,"statfs":{"total":289910292480,"available":287945232384,"internally_reserved":0,"allocated":1175433216,"data_stored":1174579791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":789564423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,10,5,12,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4165,"num_objects_misplaced":3943,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4497,"num_write_kb":17988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.586519"},"pg_stats":[{"pgid":"2.f","version":"41'8721","reported_seq":8789,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.776703+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:45.776703+0000","last_peered":"2024-09-11T01:17:45.776703+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:45.776703+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8721,"num_write_kb":34884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4598},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8834","reported_seq":8909,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.162903+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:45.162903+0000","last_peered":"2024-09-11T01:17:45.162903+0000","last_clean":"2024-09-11T01:17:45.162903+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:45.162903+0000","last_undegraded":"2024-09-11T01:17:45.162903+0000","last_fullsized":"2024-09-11T01:17:45.162903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36160454,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8650","reported_seq":8728,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.137115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:45.137115+0000","last_peered":"2024-09-11T01:17:45.137115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:45.137115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8650,"log_dups_size":0,"ondisk_log_size":8650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6711,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8762","reported_seq":8802,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.902948+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:44.902948+0000","last_peered":"2024-09-11T01:17:44.902948+0000","last_clean":"2024-09-11T01:17:44.902948+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:44.902948+0000","last_undegraded":"2024-09-11T01:17:44.902948+0000","last_fullsized":"2024-09-11T01:17:44.902948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35889152,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":35048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8745","reported_seq":8861,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.775910+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:45.775910+0000","last_peered":"2024-09-11T01:17:45.775910+0000","last_clean":"2024-09-11T01:17:45.775910+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:45.775910+0000","last_undegraded":"2024-09-11T01:17:45.775910+0000","last_fullsized":"2024-09-11T01:17:45.775910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8745,"log_dups_size":0,"ondisk_log_size":8745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35791331,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5543,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8785","reported_seq":8912,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.763424+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:45.763424+0000","last_peered":"2024-09-11T01:17:45.763424+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:45.763424+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35963363,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5273,"num_write_kb":21092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4666},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9020","reported_seq":9148,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.724436+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:45.724436+0000","last_peered":"2024-09-11T01:17:45.724436+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:45.724436+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9020,"log_dups_size":0,"ondisk_log_size":9020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36922310,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5773,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8825","reported_seq":8939,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.754156+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:45.754156+0000","last_peered":"2024-09-11T01:17:45.754156+0000","last_clean":"2024-09-11T01:17:45.754156+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:45.754156+0000","last_undegraded":"2024-09-11T01:17:45.754156+0000","last_fullsized":"2024-09-11T01:17:45.754156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8825,"log_dups_size":0,"ondisk_log_size":8825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36139008,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8863","reported_seq":17201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:48.341653+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:48.341653+0000","last_peered":"2024-09-11T01:17:48.341653+0000","last_clean":"2024-09-11T01:17:48.341653+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:48.341653+0000","last_undegraded":"2024-09-11T01:17:48.341653+0000","last_fullsized":"2024-09-11T01:17:48.341653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8863,"log_dups_size":0,"ondisk_log_size":8863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274659,"num_objects":8857,"num_object_clones":0,"num_object_copies":17714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8907","reported_seq":9031,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.747957+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:45.747957+0000","last_peered":"2024-09-11T01:17:45.747957+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:45.747957+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36450304,"num_objects":8899,"num_object_clones":0,"num_object_copies":17798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5271,"num_write_kb":21084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4794},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8637","reported_seq":8726,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.074913+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:45.074913+0000","last_peered":"2024-09-11T01:17:45.074913+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:45.074913+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35357155,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8922","reported_seq":9010,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.128408+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:45.128408+0000","last_peered":"2024-09-11T01:17:45.128408+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:45.128408+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8922,"log_dups_size":0,"ondisk_log_size":8922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36519936,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4739}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8841","reported_seq":9274,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:45.797624+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:45.797624+0000","last_peered":"2024-09-11T01:17:45.797624+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:45.797624+0000","last_undegraded":"2024-09-11T01:17:45.797624+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8841,"log_dups_size":0,"ondisk_log_size":8841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5531,"num_write_kb":22124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4829},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9002","reported_seq":9131,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.664033+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:45.664033+0000","last_peered":"2024-09-11T01:17:45.664033+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:45.664033+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4802},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8890","reported_seq":9018,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.639099+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:45.639099+0000","last_peered":"2024-09-11T01:17:45.639099+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:45.639099+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8890,"log_dups_size":0,"ondisk_log_size":8890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36397056,"num_objects":8886,"num_object_clones":0,"num_object_copies":17772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5425,"num_write_kb":21700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4738},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8741","reported_seq":8871,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.756614+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:45.756614+0000","last_peered":"2024-09-11T01:17:45.756614+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:45.756614+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8741,"log_dups_size":0,"ondisk_log_size":8741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35754950,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5398,"num_write_kb":21592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4577},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577798878,"num_objects":141073,"num_object_clones":0,"num_object_copies":282146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":141073,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":100208,"num_write_kb":400832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4280,"num_bytes_recovered":17523171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1176702976,"data_stored":1176551230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141145,"ondisk_log_size":141145,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661772,"kb_used_data":413500,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93710068,"statfs":{"total":96636764160,"available":95959109632,"internally_reserved":0,"allocated":423424000,"data_stored":423156695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457392,"kb_used_data":245216,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93914448,"statfs":{"total":96636764160,"available":96168394752,"internally_reserved":0,"allocated":251101184,"data_stored":250786446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799840,"kb_used_data":489168,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93572000,"statfs":{"total":96636764160,"available":95817728000,"internally_reserved":0,"allocated":500908032,"data_stored":500636650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":501321728,"data_stored":501285598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252010496,"data_stored":251931010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423370752,"data_stored":423334622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:49.753 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:50.344 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:50.346 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:50.352 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 194 20 143178 143158 2.88193 1.65625 0.0604359 0.0271006 2024-09-11T01:17:50.369 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-11T01:17:48.477018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577798897,"num_objects":141075,"num_object_clones":0,"num_object_copies":282150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":141075,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":100210,"num_write_kb":400834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4281,"num_bytes_recovered":17523190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141148,"ondisk_log_size":141148,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1919004,"kb_used_data":1147884,"kb_used_omap":12,"kb_used_meta":771059,"kb_avail":281196516,"statfs":{"total":289910292480,"available":287945232384,"internally_reserved":0,"allocated":1175433216,"data_stored":1174579791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":789564423},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,10,5,12,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4165,"num_objects_misplaced":3943,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4497,"num_write_kb":17988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.586519"},"pg_stats":[{"pgid":"2.f","version":"41'8721","reported_seq":8789,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.776703+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:45.776703+0000","last_peered":"2024-09-11T01:17:45.776703+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:45.776703+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8721,"num_write_kb":34884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4598},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8834","reported_seq":8909,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.162903+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:45.162903+0000","last_peered":"2024-09-11T01:17:45.162903+0000","last_clean":"2024-09-11T01:17:45.162903+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:45.162903+0000","last_undegraded":"2024-09-11T01:17:45.162903+0000","last_fullsized":"2024-09-11T01:17:45.162903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8834,"log_dups_size":0,"ondisk_log_size":8834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36160454,"num_objects":8830,"num_object_clones":0,"num_object_copies":17660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8830,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6932,"num_write_kb":27728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8650","reported_seq":8728,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.137115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:45.137115+0000","last_peered":"2024-09-11T01:17:45.137115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:45.137115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8650,"log_dups_size":0,"ondisk_log_size":8650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6711,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8762","reported_seq":8802,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.902948+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:44.902948+0000","last_peered":"2024-09-11T01:17:44.902948+0000","last_clean":"2024-09-11T01:17:44.902948+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:44.902948+0000","last_undegraded":"2024-09-11T01:17:44.902948+0000","last_fullsized":"2024-09-11T01:17:44.902948+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35889152,"num_objects":8762,"num_object_clones":0,"num_object_copies":17524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8762,"num_write_kb":35048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8745","reported_seq":8861,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.775910+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:45.775910+0000","last_peered":"2024-09-11T01:17:45.775910+0000","last_clean":"2024-09-11T01:17:45.775910+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:45.775910+0000","last_undegraded":"2024-09-11T01:17:45.775910+0000","last_fullsized":"2024-09-11T01:17:45.775910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8745,"log_dups_size":0,"ondisk_log_size":8745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35791331,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5543,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8785","reported_seq":8912,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.763424+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:45.763424+0000","last_peered":"2024-09-11T01:17:45.763424+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:45.763424+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35963363,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5273,"num_write_kb":21092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4666},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9020","reported_seq":9148,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.724436+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:45.724436+0000","last_peered":"2024-09-11T01:17:45.724436+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:45.724436+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9020,"log_dups_size":0,"ondisk_log_size":9020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36922310,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5773,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8825","reported_seq":8939,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.754156+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:45.754156+0000","last_peered":"2024-09-11T01:17:45.754156+0000","last_clean":"2024-09-11T01:17:45.754156+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:45.754156+0000","last_undegraded":"2024-09-11T01:17:45.754156+0000","last_fullsized":"2024-09-11T01:17:45.754156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8825,"log_dups_size":0,"ondisk_log_size":8825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36139008,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8863","reported_seq":17201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:48.341653+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:48.341653+0000","last_peered":"2024-09-11T01:17:48.341653+0000","last_clean":"2024-09-11T01:17:48.341653+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:48.341653+0000","last_undegraded":"2024-09-11T01:17:48.341653+0000","last_fullsized":"2024-09-11T01:17:48.341653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8863,"log_dups_size":0,"ondisk_log_size":8863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274659,"num_objects":8857,"num_object_clones":0,"num_object_copies":17714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8907","reported_seq":9031,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.747957+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:45.747957+0000","last_peered":"2024-09-11T01:17:45.747957+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:45.747957+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36450304,"num_objects":8899,"num_object_clones":0,"num_object_copies":17798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5271,"num_write_kb":21084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4794},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8637","reported_seq":8726,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.074913+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:45.074913+0000","last_peered":"2024-09-11T01:17:45.074913+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:45.074913+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35357155,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6406,"num_write_kb":25624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4606}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'8922","reported_seq":9010,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.128408+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:45.128408+0000","last_peered":"2024-09-11T01:17:45.128408+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:45.128408+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":8922,"log_dups_size":0,"ondisk_log_size":8922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36519936,"num_objects":8916,"num_object_clones":0,"num_object_copies":17832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4739}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8841","reported_seq":9274,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:45.797624+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:45.797624+0000","last_peered":"2024-09-11T01:17:45.797624+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:45.797624+0000","last_undegraded":"2024-09-11T01:17:45.797624+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8841,"log_dups_size":0,"ondisk_log_size":8841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5531,"num_write_kb":22124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4829},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9002","reported_seq":9131,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.664033+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:45.664033+0000","last_peered":"2024-09-11T01:17:45.664033+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:45.664033+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4802},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8890","reported_seq":9018,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.639099+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:45.639099+0000","last_peered":"2024-09-11T01:17:45.639099+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:45.639099+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8890,"log_dups_size":0,"ondisk_log_size":8890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36397056,"num_objects":8886,"num_object_clones":0,"num_object_copies":17772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5425,"num_write_kb":21700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4738},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8741","reported_seq":8871,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.756614+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:45.756614+0000","last_peered":"2024-09-11T01:17:45.756614+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:45.756614+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8741,"log_dups_size":0,"ondisk_log_size":8741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35754950,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5398,"num_write_kb":21592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4577},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577798878,"num_objects":141073,"num_object_clones":0,"num_object_copies":282146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":141073,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":100208,"num_write_kb":400832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4280,"num_bytes_recovered":17523171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1176702976,"data_stored":1176551230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141145,"ondisk_log_size":141145,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661772,"kb_used_data":413500,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93710068,"statfs":{"total":96636764160,"available":95959109632,"internally_reserved":0,"allocated":423424000,"data_stored":423156695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457392,"kb_used_data":245216,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93914448,"statfs":{"total":96636764160,"available":96168394752,"internally_reserved":0,"allocated":251101184,"data_stored":250786446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799840,"kb_used_data":489168,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93572000,"statfs":{"total":96636764160,"available":95817728000,"internally_reserved":0,"allocated":500908032,"data_stored":500636650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":501321728,"data_stored":501285598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252010496,"data_stored":251931010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423370752,"data_stored":423334622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:50.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:50.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'8721", 'reported_seq': 8789, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.776703+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:45.776703+0000', 'last_peered': '2024-09-11T01:17:45.776703+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:45.776703+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 8721, 'log_dups_size': 0, 'ondisk_log_size': 8721, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35721216, 'num_objects': 8721, 'num_object_clones': 0, 'num_object_copies': 17442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8721, 'num_write_kb': 34884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4598}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:50.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'8650", 'reported_seq': 8728, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.137115+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:45.137115+0000', 'last_peered': '2024-09-11T01:17:45.137115+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:45.137115+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 8650, 'log_dups_size': 0, 'ondisk_log_size': 8650, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35414016, 'num_objects': 8646, 'num_object_clones': 0, 'num_object_copies': 17292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6711, 'num_write_kb': 26844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 4641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:50.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:50.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'8785", 'reported_seq': 8912, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.763424+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:45.763424+0000', 'last_peered': '2024-09-11T01:17:45.763424+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:45.763424+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 8785, 'log_dups_size': 0, 'ondisk_log_size': 8785, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35963363, 'num_objects': 8781, 'num_object_clones': 0, 'num_object_copies': 17562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8781, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5273, 'num_write_kb': 21092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4666}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9020", 'reported_seq': 9148, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.724436+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:45.724436+0000', 'last_peered': '2024-09-11T01:17:45.724436+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:45.724436+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9020, 'log_dups_size': 0, 'ondisk_log_size': 9020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36922310, 'num_objects': 9016, 'num_object_clones': 0, 'num_object_copies': 18032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9016, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5773, 'num_write_kb': 23092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4733}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'8907", 'reported_seq': 9031, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.747957+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:45.747957+0000', 'last_peered': '2024-09-11T01:17:45.747957+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:45.747957+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 8907, 'log_dups_size': 0, 'ondisk_log_size': 8907, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36450304, 'num_objects': 8899, 'num_object_clones': 0, 'num_object_copies': 17798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8899, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5271, 'num_write_kb': 21084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4794}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'8637", 'reported_seq': 8726, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.074913+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:45.074913+0000', 'last_peered': '2024-09-11T01:17:45.074913+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:45.074913+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 8637, 'log_dups_size': 0, 'ondisk_log_size': 8637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35357155, 'num_objects': 8633, 'num_object_clones': 0, 'num_object_copies': 17266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8633, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6406, 'num_write_kb': 25624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 4606}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'8922", 'reported_seq': 9010, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.128408+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:45.128408+0000', 'last_peered': '2024-09-11T01:17:45.128408+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:45.128408+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 8922, 'log_dups_size': 0, 'ondisk_log_size': 8922, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36519936, 'num_objects': 8916, 'num_object_clones': 0, 'num_object_copies': 17832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6559, 'num_write_kb': 26236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4739}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'8841", 'reported_seq': 9274, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:45.797624+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:17:45.797624+0000', 'last_peered': '2024-09-11T01:17:45.797624+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:45.797624+0000', 'last_undegraded': '2024-09-11T01:17:45.797624+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 8841, 'log_dups_size': 0, 'ondisk_log_size': 8841, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36204544, 'num_objects': 8839, 'num_object_clones': 0, 'num_object_copies': 17678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4010, 'num_objects_unfound': 0, 'num_objects_dirty': 8839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5531, 'num_write_kb': 22124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 154, 'num_bytes_recovered': 630784, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4829}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:50.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9002", 'reported_seq': 9131, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.664033+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:45.664033+0000', 'last_peered': '2024-09-11T01:17:45.664033+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:45.664033+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9002, 'log_dups_size': 0, 'ondisk_log_size': 9002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36839424, 'num_objects': 8994, 'num_object_clones': 0, 'num_object_copies': 17988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8994, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5395, 'num_write_kb': 21580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4802}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:50.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'8890", 'reported_seq': 9018, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.639099+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:45.639099+0000', 'last_peered': '2024-09-11T01:17:45.639099+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:45.639099+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 8890, 'log_dups_size': 0, 'ondisk_log_size': 8890, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36397056, 'num_objects': 8886, 'num_object_clones': 0, 'num_object_copies': 17772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8886, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5425, 'num_write_kb': 21700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4738}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:50.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'8741", 'reported_seq': 8871, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:45.756614+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:45.756614+0000', 'last_peered': '2024-09-11T01:17:45.756614+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:45.756614+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 8741, 'log_dups_size': 0, 'ondisk_log_size': 8741, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35754950, 'num_objects': 8731, 'num_object_clones': 0, 'num_object_copies': 17462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8731, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5398, 'num_write_kb': 21592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4577}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:50.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:51.127 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:51.127 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/283650 objects degraded (14.575%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":271,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":141825,"data_bytes":580870897,"bytes_used":1974063104,"bytes_avail":287936229376,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":283650,"degraded_ratio":0.14575356953992596,"misplaced_objects":4010,"misplaced_total":283650,"misplaced_ratio":0.014137140842587697,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":52537,"recovering_keys_per_sec":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"write_bytes_sec":1762386,"read_op_per_sec":0,"write_op_per_sec":430},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:51.153 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:51.153 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:51.355 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 195 20 143607 143587 2.8757 1.67578 0.0272902 0.0271589 2024-09-11T01:17:51.715 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:51.717 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:51.743 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-11T01:17:50.477490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580870897,"num_objects":141825,"num_object_clones":0,"num_object_copies":283650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":141825,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":100960,"num_write_kb":403834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4281,"num_bytes_recovered":17523190,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141898,"ondisk_log_size":141898,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1927796,"kb_used_data":1152516,"kb_used_omap":12,"kb_used_meta":775219,"kb_avail":281187724,"statfs":{"total":289910292480,"available":287936229376,"internally_reserved":0,"allocated":1180176384,"data_stored":1179322959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":793824263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,9,4,12,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21159936,"num_objects":5166,"num_object_clones":0,"num_object_copies":10332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4165,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":5166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5166,"num_write_kb":20664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006409"},"pg_stats":[{"pgid":"2.f","version":"41'8721","reported_seq":8789,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.776703+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:45.776703+0000","last_peered":"2024-09-11T01:17:45.776703+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:45.776703+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8721,"log_dups_size":0,"ondisk_log_size":8721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8721,"num_write_kb":34884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4598},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8984","reported_seq":9059,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.083676+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:50.083676+0000","last_peered":"2024-09-11T01:17:50.083676+0000","last_clean":"2024-09-11T01:17:50.083676+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:50.083676+0000","last_undegraded":"2024-09-11T01:17:50.083676+0000","last_fullsized":"2024-09-11T01:17:50.083676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8984,"log_dups_size":0,"ondisk_log_size":8984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36774854,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7082,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8802","reported_seq":8879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.127824+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:50.127824+0000","last_peered":"2024-09-11T01:17:50.127824+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:50.127824+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6863,"num_write_kb":27452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8924","reported_seq":8964,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.083711+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:50.083711+0000","last_peered":"2024-09-11T01:17:50.083711+0000","last_clean":"2024-09-11T01:17:50.083711+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:50.083711+0000","last_undegraded":"2024-09-11T01:17:50.083711+0000","last_fullsized":"2024-09-11T01:17:50.083711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8924,"log_dups_size":0,"ondisk_log_size":8924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8924,"num_write_kb":35696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8745","reported_seq":8861,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.775910+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:45.775910+0000","last_peered":"2024-09-11T01:17:45.775910+0000","last_clean":"2024-09-11T01:17:45.775910+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:45.775910+0000","last_undegraded":"2024-09-11T01:17:45.775910+0000","last_fullsized":"2024-09-11T01:17:45.775910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8745,"log_dups_size":0,"ondisk_log_size":8745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35791331,"num_objects":8739,"num_object_clones":0,"num_object_copies":17478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5543,"num_write_kb":22172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8785","reported_seq":8912,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.763424+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:45.763424+0000","last_peered":"2024-09-11T01:17:45.763424+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:45.763424+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8785,"log_dups_size":0,"ondisk_log_size":8785,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35963363,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5273,"num_write_kb":21092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4666},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9020","reported_seq":9148,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.724436+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:45.724436+0000","last_peered":"2024-09-11T01:17:45.724436+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:45.724436+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9020,"log_dups_size":0,"ondisk_log_size":9020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36922310,"num_objects":9016,"num_object_clones":0,"num_object_copies":18032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9016,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5773,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8825","reported_seq":8939,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:45.754156+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:45.754156+0000","last_peered":"2024-09-11T01:17:45.754156+0000","last_clean":"2024-09-11T01:17:45.754156+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:45.754156+0000","last_undegraded":"2024-09-11T01:17:45.754156+0000","last_fullsized":"2024-09-11T01:17:45.754156+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8825,"log_dups_size":0,"ondisk_log_size":8825,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36139008,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5778,"num_write_kb":23112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8863","reported_seq":17201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:48.341653+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:48.341653+0000","last_peered":"2024-09-11T01:17:48.341653+0000","last_clean":"2024-09-11T01:17:48.341653+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:48.341653+0000","last_undegraded":"2024-09-11T01:17:48.341653+0000","last_fullsized":"2024-09-11T01:17:48.341653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8863,"log_dups_size":0,"ondisk_log_size":8863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274659,"num_objects":8857,"num_object_clones":0,"num_object_copies":17714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8857,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'8907","reported_seq":9031,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.747957+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:45.747957+0000","last_peered":"2024-09-11T01:17:45.747957+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:45.747957+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36450304,"num_objects":8899,"num_object_clones":0,"num_object_copies":17798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5271,"num_write_kb":21084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4794},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8790","reported_seq":8879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.977048+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:49.977048+0000","last_peered":"2024-09-11T01:17:49.977048+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:49.977048+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8790,"log_dups_size":0,"ondisk_log_size":8790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983843,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4759}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9055","reported_seq":9141,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.146467+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:50.146467+0000","last_peered":"2024-09-11T01:17:50.146467+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:50.146467+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9055,"log_dups_size":0,"ondisk_log_size":9055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6692,"num_write_kb":26768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8841","reported_seq":9274,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:45.797624+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:45.797624+0000","last_peered":"2024-09-11T01:17:45.797624+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:45.797624+0000","last_undegraded":"2024-09-11T01:17:45.797624+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8841,"log_dups_size":0,"ondisk_log_size":8841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5531,"num_write_kb":22124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":630784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4829},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9002","reported_seq":9131,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.664033+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:45.664033+0000","last_peered":"2024-09-11T01:17:45.664033+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:45.664033+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36839424,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":21580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4802},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'8890","reported_seq":9018,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.639099+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:45.639099+0000","last_peered":"2024-09-11T01:17:45.639099+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:45.639099+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":8890,"log_dups_size":0,"ondisk_log_size":8890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36397056,"num_objects":8886,"num_object_clones":0,"num_object_copies":17772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5425,"num_write_kb":21700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4738},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8741","reported_seq":8871,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:45.756614+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:45.756614+0000","last_peered":"2024-09-11T01:17:45.756614+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:45.756614+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8741,"log_dups_size":0,"ondisk_log_size":8741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35754950,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5398,"num_write_kb":21592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4577},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":580870878,"num_objects":141823,"num_object_clones":0,"num_object_copies":283646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":4010,"num_objects_unfound":0,"num_objects_dirty":141823,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":100958,"num_write_kb":403832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4280,"num_bytes_recovered":17523171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1181462528,"data_stored":1181310782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141895,"ondisk_log_size":141895,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670564,"kb_used_data":418132,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93701276,"statfs":{"total":96636764160,"available":95950106624,"internally_reserved":0,"allocated":428167168,"data_stored":427899863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789983,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457392,"kb_used_data":245216,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93914448,"statfs":{"total":96636764160,"available":96168394752,"internally_reserved":0,"allocated":251101184,"data_stored":250786446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799840,"kb_used_data":489168,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93572000,"statfs":{"total":96636764160,"available":95817728000,"internally_reserved":0,"allocated":500908032,"data_stored":500636650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":501321728,"data_stored":501285598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252010496,"data_stored":251931010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428130304,"data_stored":428094174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:52.355 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 196 20 144068 144048 2.87022 1.80078 0.0352548 0.0272125 2024-09-11T01:17:53.355 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 197 20 144540 144520 2.86501 1.84375 0.105697 0.0272612 2024-09-11T01:17:54.355 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 198 19 145006 144987 2.85975 1.82422 0.0860512 0.0273114 2024-09-11T01:17:54.745 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:55.315 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:55.315 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:55.340 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-11T01:17:54.479061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587268849,"num_objects":143387,"num_object_clones":0,"num_object_copies":286774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3940,"num_objects_unfound":0,"num_objects_dirty":143387,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":102522,"num_write_kb":410082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4351,"num_bytes_recovered":17809910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143460,"ondisk_log_size":143460,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1933336,"kb_used_data":1166376,"kb_used_omap":12,"kb_used_meta":766899,"kb_avail":281182184,"statfs":{"total":289910292480,"available":287930556416,"internally_reserved":0,"allocated":1194369024,"data_stored":1193515599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":785304583},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,8,12,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19472384,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-131,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":536576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007078"},"pg_stats":[{"pgid":"2.f","version":"41'8865","reported_seq":8933,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.869761+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:50.869761+0000","last_peered":"2024-09-11T01:17:50.869761+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:50.869761+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8865,"log_dups_size":0,"ondisk_log_size":8865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36311040,"num_objects":8865,"num_object_clones":0,"num_object_copies":17730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8865,"num_write_kb":35460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8984","reported_seq":9059,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.083676+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:50.083676+0000","last_peered":"2024-09-11T01:17:50.083676+0000","last_clean":"2024-09-11T01:17:50.083676+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:50.083676+0000","last_undegraded":"2024-09-11T01:17:50.083676+0000","last_fullsized":"2024-09-11T01:17:50.083676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8984,"log_dups_size":0,"ondisk_log_size":8984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36774854,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7082,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8802","reported_seq":8879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.127824+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:50.127824+0000","last_peered":"2024-09-11T01:17:50.127824+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:50.127824+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6863,"num_write_kb":27452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8924","reported_seq":8964,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.083711+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:50.083711+0000","last_peered":"2024-09-11T01:17:50.083711+0000","last_clean":"2024-09-11T01:17:50.083711+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:50.083711+0000","last_undegraded":"2024-09-11T01:17:50.083711+0000","last_fullsized":"2024-09-11T01:17:50.083711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8924,"log_dups_size":0,"ondisk_log_size":8924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8924,"num_write_kb":35696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8896","reported_seq":9012,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.861696+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:50.861696+0000","last_peered":"2024-09-11T01:17:50.861696+0000","last_clean":"2024-09-11T01:17:50.861696+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:50.861696+0000","last_undegraded":"2024-09-11T01:17:50.861696+0000","last_fullsized":"2024-09-11T01:17:50.861696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8896,"log_dups_size":0,"ondisk_log_size":8896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409827,"num_objects":8890,"num_object_clones":0,"num_object_copies":17780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5694,"num_write_kb":22776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8929","reported_seq":9057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.761298+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:50.761298+0000","last_peered":"2024-09-11T01:17:50.761298+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:50.761298+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8929,"log_dups_size":0,"ondisk_log_size":8929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36553187,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5417,"num_write_kb":21668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9176","reported_seq":9304,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.789618+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:50.789618+0000","last_peered":"2024-09-11T01:17:50.789618+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:50.789618+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9176,"log_dups_size":0,"ondisk_log_size":9176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37561286,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5929,"num_write_kb":23716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4889},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8963","reported_seq":9076,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.850613+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:50.850613+0000","last_peered":"2024-09-11T01:17:50.850613+0000","last_clean":"2024-09-11T01:17:50.850613+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:50.850613+0000","last_undegraded":"2024-09-11T01:17:50.850613+0000","last_fullsized":"2024-09-11T01:17:50.850613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8963,"log_dups_size":0,"ondisk_log_size":8963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704256,"num_objects":8961,"num_object_clones":0,"num_object_copies":17922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5916,"num_write_kb":23664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8987","reported_seq":17325,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:53.436967+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:53.436967+0000","last_peered":"2024-09-11T01:17:53.436967+0000","last_clean":"2024-09-11T01:17:53.436967+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:53.436967+0000","last_undegraded":"2024-09-11T01:17:53.436967+0000","last_fullsized":"2024-09-11T01:17:53.436967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8987,"log_dups_size":0,"ondisk_log_size":8987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782563,"num_objects":8981,"num_object_clones":0,"num_object_copies":17962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9037","reported_seq":9161,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.818825+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:50.818825+0000","last_peered":"2024-09-11T01:17:50.818825+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:50.818825+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9037,"log_dups_size":0,"ondisk_log_size":9037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5401,"num_write_kb":21604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8790","reported_seq":8879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.977048+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:49.977048+0000","last_peered":"2024-09-11T01:17:49.977048+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:49.977048+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8790,"log_dups_size":0,"ondisk_log_size":8790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983843,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4759}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9055","reported_seq":9141,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.146467+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:50.146467+0000","last_peered":"2024-09-11T01:17:50.146467+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:50.146467+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9055,"log_dups_size":0,"ondisk_log_size":9055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6692,"num_write_kb":26768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8986","reported_seq":9559,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:50.900407+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:50.900407+0000","last_peered":"2024-09-11T01:17:50.900407+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:50.900407+0000","last_undegraded":"2024-09-11T01:17:50.900407+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8986,"log_dups_size":0,"ondisk_log_size":8986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36798464,"num_objects":8984,"num_object_clones":0,"num_object_copies":17968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3940,"num_objects_unfound":0,"num_objects_dirty":8984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5676,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5044},{"shards":"0,2","objects":3940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9139","reported_seq":9268,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.842755+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:50.842755+0000","last_peered":"2024-09-11T01:17:50.842755+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:50.842755+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4939},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9053","reported_seq":9181,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.822564+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:50.822564+0000","last_peered":"2024-09-11T01:17:50.822564+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:50.822564+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4901},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8871","reported_seq":9001,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.788769+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:50.788769+0000","last_peered":"2024-09-11T01:17:50.788769+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:50.788769+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8871,"log_dups_size":0,"ondisk_log_size":8871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36287430,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5528,"num_write_kb":22112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4707},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587268830,"num_objects":143385,"num_object_clones":0,"num_object_copies":286770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3940,"num_objects_unfound":0,"num_objects_dirty":143385,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":102520,"num_write_kb":410080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4350,"num_bytes_recovered":17809891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1195945984,"data_stored":1195794238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143457,"ondisk_log_size":143457,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670564,"kb_used_data":418132,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93701276,"statfs":{"total":96636764160,"available":95950106624,"internally_reserved":0,"allocated":428167168,"data_stored":427899863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789984,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468652,"kb_used_data":252316,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93903188,"statfs":{"total":96636764160,"available":96156864512,"internally_reserved":0,"allocated":258371584,"data_stored":258056846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794120,"kb_used_data":495928,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93577720,"statfs":{"total":96636764160,"available":95823585280,"internally_reserved":0,"allocated":507830272,"data_stored":507558890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":508567552,"data_stored":508531422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259248128,"data_stored":259168642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428130304,"data_stored":428094174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:55.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:55.359 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 199 20 145448 145428 2.85404 1.72266 0.0301964 0.0273662 2024-09-11T01:17:55.891 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:55.891 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:55.919 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-11T01:17:54.479061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":587268849,"num_objects":143387,"num_object_clones":0,"num_object_copies":286774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3940,"num_objects_unfound":0,"num_objects_dirty":143387,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":102522,"num_write_kb":410082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4351,"num_bytes_recovered":17809910,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143460,"ondisk_log_size":143460,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1933336,"kb_used_data":1166376,"kb_used_omap":12,"kb_used_meta":766899,"kb_avail":281182184,"statfs":{"total":289910292480,"available":287930556416,"internally_reserved":0,"allocated":1194369024,"data_stored":1193515599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":785304583},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2,8,12,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19472384,"num_objects":4754,"num_object_clones":0,"num_object_copies":9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-131,"num_objects_unfound":0,"num_objects_dirty":4754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":19016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":131,"num_bytes_recovered":536576,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007078"},"pg_stats":[{"pgid":"2.f","version":"41'8865","reported_seq":8933,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.869761+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:50.869761+0000","last_peered":"2024-09-11T01:17:50.869761+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:50.869761+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8865,"log_dups_size":0,"ondisk_log_size":8865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36311040,"num_objects":8865,"num_object_clones":0,"num_object_copies":17730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8865,"num_write_kb":35460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'8984","reported_seq":9059,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.083676+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:50.083676+0000","last_peered":"2024-09-11T01:17:50.083676+0000","last_clean":"2024-09-11T01:17:50.083676+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:50.083676+0000","last_undegraded":"2024-09-11T01:17:50.083676+0000","last_fullsized":"2024-09-11T01:17:50.083676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":8984,"log_dups_size":0,"ondisk_log_size":8984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36774854,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7082,"num_write_kb":28328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8802","reported_seq":8879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.127824+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:50.127824+0000","last_peered":"2024-09-11T01:17:50.127824+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:50.127824+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8802,"log_dups_size":0,"ondisk_log_size":8802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6863,"num_write_kb":27452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'8924","reported_seq":8964,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.083711+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:50.083711+0000","last_peered":"2024-09-11T01:17:50.083711+0000","last_clean":"2024-09-11T01:17:50.083711+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:50.083711+0000","last_undegraded":"2024-09-11T01:17:50.083711+0000","last_fullsized":"2024-09-11T01:17:50.083711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":8924,"log_dups_size":0,"ondisk_log_size":8924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8924,"num_write_kb":35696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'8896","reported_seq":9012,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.861696+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:50.861696+0000","last_peered":"2024-09-11T01:17:50.861696+0000","last_clean":"2024-09-11T01:17:50.861696+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:50.861696+0000","last_undegraded":"2024-09-11T01:17:50.861696+0000","last_fullsized":"2024-09-11T01:17:50.861696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":8896,"log_dups_size":0,"ondisk_log_size":8896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36409827,"num_objects":8890,"num_object_clones":0,"num_object_copies":17780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8890,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5694,"num_write_kb":22776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'8929","reported_seq":9057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.761298+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:50.761298+0000","last_peered":"2024-09-11T01:17:50.761298+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:50.761298+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":8929,"log_dups_size":0,"ondisk_log_size":8929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36553187,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5417,"num_write_kb":21668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4810},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9176","reported_seq":9304,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.789618+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:50.789618+0000","last_peered":"2024-09-11T01:17:50.789618+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:50.789618+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9176,"log_dups_size":0,"ondisk_log_size":9176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37561286,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5929,"num_write_kb":23716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4889},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'8963","reported_seq":9076,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:50.850613+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:50.850613+0000","last_peered":"2024-09-11T01:17:50.850613+0000","last_clean":"2024-09-11T01:17:50.850613+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:50.850613+0000","last_undegraded":"2024-09-11T01:17:50.850613+0000","last_fullsized":"2024-09-11T01:17:50.850613+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":8963,"log_dups_size":0,"ondisk_log_size":8963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36704256,"num_objects":8961,"num_object_clones":0,"num_object_copies":17922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5916,"num_write_kb":23664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8987","reported_seq":17325,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:53.436967+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:53.436967+0000","last_peered":"2024-09-11T01:17:53.436967+0000","last_clean":"2024-09-11T01:17:53.436967+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:53.436967+0000","last_undegraded":"2024-09-11T01:17:53.436967+0000","last_fullsized":"2024-09-11T01:17:53.436967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8987,"log_dups_size":0,"ondisk_log_size":8987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782563,"num_objects":8981,"num_object_clones":0,"num_object_copies":17962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9037","reported_seq":9161,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.818825+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:50.818825+0000","last_peered":"2024-09-11T01:17:50.818825+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:50.818825+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9037,"log_dups_size":0,"ondisk_log_size":9037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5401,"num_write_kb":21604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4924},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8790","reported_seq":8879,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.977048+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:49.977048+0000","last_peered":"2024-09-11T01:17:49.977048+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:49.977048+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8790,"log_dups_size":0,"ondisk_log_size":8790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35983843,"num_objects":8786,"num_object_clones":0,"num_object_copies":17572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4759}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9055","reported_seq":9141,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.146467+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:50.146467+0000","last_peered":"2024-09-11T01:17:50.146467+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:50.146467+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9055,"log_dups_size":0,"ondisk_log_size":9055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6692,"num_write_kb":26768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":4872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'8986","reported_seq":9559,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:50.900407+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:50.900407+0000","last_peered":"2024-09-11T01:17:50.900407+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:50.900407+0000","last_undegraded":"2024-09-11T01:17:50.900407+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":8986,"log_dups_size":0,"ondisk_log_size":8986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36798464,"num_objects":8984,"num_object_clones":0,"num_object_copies":17968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3940,"num_objects_unfound":0,"num_objects_dirty":8984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5676,"num_write_kb":22704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5044},{"shards":"0,2","objects":3940}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9139","reported_seq":9268,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.842755+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:50.842755+0000","last_peered":"2024-09-11T01:17:50.842755+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:50.842755+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5532,"num_write_kb":22128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4939},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9053","reported_seq":9181,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.822564+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:50.822564+0000","last_peered":"2024-09-11T01:17:50.822564+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:50.822564+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4901},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'8871","reported_seq":9001,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:50.788769+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:50.788769+0000","last_peered":"2024-09-11T01:17:50.788769+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:50.788769+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":8871,"log_dups_size":0,"ondisk_log_size":8871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36287430,"num_objects":8861,"num_object_clones":0,"num_object_copies":17722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5528,"num_write_kb":22112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4707},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":587268830,"num_objects":143385,"num_object_clones":0,"num_object_copies":286770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3940,"num_objects_unfound":0,"num_objects_dirty":143385,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":102520,"num_write_kb":410080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4350,"num_bytes_recovered":17809891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1195945984,"data_stored":1195794238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143457,"ondisk_log_size":143457,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670564,"kb_used_data":418132,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93701276,"statfs":{"total":96636764160,"available":95950106624,"internally_reserved":0,"allocated":428167168,"data_stored":427899863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789984,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468652,"kb_used_data":252316,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93903188,"statfs":{"total":96636764160,"available":96156864512,"internally_reserved":0,"allocated":258371584,"data_stored":258056846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":794120,"kb_used_data":495928,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93577720,"statfs":{"total":96636764160,"available":95823585280,"internally_reserved":0,"allocated":507830272,"data_stored":507558890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":508567552,"data_stored":508531422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259248128,"data_stored":259168642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428130304,"data_stored":428094174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:55.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:55.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'8865", 'reported_seq': 8933, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.869761+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:50.869761+0000', 'last_peered': '2024-09-11T01:17:50.869761+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:50.869761+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 8865, 'log_dups_size': 0, 'ondisk_log_size': 8865, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36311040, 'num_objects': 8865, 'num_object_clones': 0, 'num_object_copies': 17730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8865, 'num_write_kb': 35460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4742}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'8802", 'reported_seq': 8879, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.127824+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:17:50.127824+0000', 'last_peered': '2024-09-11T01:17:50.127824+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:17:50.127824+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 8802, 'log_dups_size': 0, 'ondisk_log_size': 8802, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36036608, 'num_objects': 8798, 'num_object_clones': 0, 'num_object_copies': 17596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6863, 'num_write_kb': 27452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 4793}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'8929", 'reported_seq': 9057, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.761298+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:50.761298+0000', 'last_peered': '2024-09-11T01:17:50.761298+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:50.761298+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 8929, 'log_dups_size': 0, 'ondisk_log_size': 8929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36553187, 'num_objects': 8925, 'num_object_clones': 0, 'num_object_copies': 17850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8925, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5417, 'num_write_kb': 21668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4810}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9176", 'reported_seq': 9304, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.789618+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:50.789618+0000', 'last_peered': '2024-09-11T01:17:50.789618+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:50.789618+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9176, 'log_dups_size': 0, 'ondisk_log_size': 9176, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37561286, 'num_objects': 9172, 'num_object_clones': 0, 'num_object_copies': 18344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9172, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 5929, 'num_write_kb': 23716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4889}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'9037", 'reported_seq': 9161, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.818825+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:50.818825+0000', 'last_peered': '2024-09-11T01:17:50.818825+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:50.818825+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 9037, 'log_dups_size': 0, 'ondisk_log_size': 9037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36982784, 'num_objects': 9029, 'num_object_clones': 0, 'num_object_copies': 18058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5401, 'num_write_kb': 21604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4924}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'8790", 'reported_seq': 8879, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:49.977048+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:17:49.977048+0000', 'last_peered': '2024-09-11T01:17:49.977048+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:17:49.977048+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 8790, 'log_dups_size': 0, 'ondisk_log_size': 8790, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35983843, 'num_objects': 8786, 'num_object_clones': 0, 'num_object_copies': 17572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8786, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6559, 'num_write_kb': 26236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 4759}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:55.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'9055", 'reported_seq': 9141, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.146467+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:17:50.146467+0000', 'last_peered': '2024-09-11T01:17:50.146467+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:17:50.146467+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 9055, 'log_dups_size': 0, 'ondisk_log_size': 9055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37064704, 'num_objects': 9049, 'num_object_clones': 0, 'num_object_copies': 18098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6692, 'num_write_kb': 26768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 4872}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'8986", 'reported_seq': 9559, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:50.900407+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:17:50.900407+0000', 'last_peered': '2024-09-11T01:17:50.900407+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:50.900407+0000', 'last_undegraded': '2024-09-11T01:17:50.900407+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 8986, 'log_dups_size': 0, 'ondisk_log_size': 8986, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36798464, 'num_objects': 8984, 'num_object_clones': 0, 'num_object_copies': 17968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3940, 'num_objects_unfound': 0, 'num_objects_dirty': 8984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5676, 'num_write_kb': 22704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 224, 'num_bytes_recovered': 917504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5044}, {'shards': '0,2', 'objects': 3940}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9139", 'reported_seq': 9268, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.842755+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:50.842755+0000', 'last_peered': '2024-09-11T01:17:50.842755+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:50.842755+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9139, 'log_dups_size': 0, 'ondisk_log_size': 9139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37400576, 'num_objects': 9131, 'num_object_clones': 0, 'num_object_copies': 18262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5532, 'num_write_kb': 22128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4939}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'9053", 'reported_seq': 9181, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.822564+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:50.822564+0000', 'last_peered': '2024-09-11T01:17:50.822564+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:50.822564+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 9053, 'log_dups_size': 0, 'ondisk_log_size': 9053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37064704, 'num_objects': 9049, 'num_object_clones': 0, 'num_object_copies': 18098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5588, 'num_write_kb': 22352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4901}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:55.922 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'8871", 'reported_seq': 9001, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:50.788769+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:50.788769+0000', 'last_peered': '2024-09-11T01:17:50.788769+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:50.788769+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 8871, 'log_dups_size': 0, 'ondisk_log_size': 8871, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36287430, 'num_objects': 8861, 'num_object_clones': 0, 'num_object_copies': 17722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8861, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5528, 'num_write_kb': 22112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4707}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:55.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:56.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:17:56.354889+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0274242 2024-09-11T01:17:56.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:17:56.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 200 20 145869 145849 2.84799 1.64453 0.0311053 0.0274242 2024-09-11T01:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:56.699 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/286774 objects degraded (14.417%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":277,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143387,"data_bytes":587268849,"bytes_used":1979736064,"bytes_avail":287930556416,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":286774,"degraded_ratio":0.1441657890882716,"misplaced_objects":3940,"misplaced_total":286774,"misplaced_ratio":0.013739041893616575,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":44688,"recovering_keys_per_sec":0,"num_objects_recovered":131,"num_bytes_recovered":536576,"num_keys_recovered":0,"write_bytes_sec":1621742,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:56.724 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:56.725 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:17:57.338 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:17:57.355 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 201 20 146286 146266 2.84192 1.62891 0.102859 0.0274818 2024-09-11T01:17:57.365 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-11T01:17:56.479684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595993329,"num_objects":145517,"num_object_clones":0,"num_object_copies":291034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":145517,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":104652,"num_write_kb":418602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4419,"num_bytes_recovered":18088438,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145590,"ondisk_log_size":145590,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1957244,"kb_used_data":1177804,"kb_used_omap":12,"kb_used_meta":779379,"kb_avail":281158276,"statfs":{"total":289910292480,"available":287906074624,"internally_reserved":0,"allocated":1206071296,"data_stored":1205217871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":798084103},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,9,13,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27688960,"num_objects":6760,"num_object_clones":0,"num_object_copies":13520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-199,"num_objects_unfound":0,"num_objects_dirty":6760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6760,"num_write_kb":27040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007219"},"pg_stats":[{"pgid":"2.f","version":"41'9009","reported_seq":9075,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.896755+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:55.896755+0000","last_peered":"2024-09-11T01:17:55.896755+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:55.896755+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9009,"num_write_kb":36036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9130","reported_seq":9205,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.073106+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:17:55.073106+0000","last_peered":"2024-09-11T01:17:55.073106+0000","last_clean":"2024-09-11T01:17:55.073106+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:17:55.073106+0000","last_undegraded":"2024-09-11T01:17:55.073106+0000","last_fullsized":"2024-09-11T01:17:55.073106+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9130,"log_dups_size":0,"ondisk_log_size":9130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37372870,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7228,"num_write_kb":28912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'8925","reported_seq":9003,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.068719+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:17:55.068719+0000","last_peered":"2024-09-11T01:17:55.068719+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:17:55.068719+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":8925,"log_dups_size":0,"ondisk_log_size":8925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36540416,"num_objects":8921,"num_object_clones":0,"num_object_copies":17842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6986,"num_write_kb":27944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":4916}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9062","reported_seq":9102,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.945064+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:17:54.945064+0000","last_peered":"2024-09-11T01:17:54.945064+0000","last_clean":"2024-09-11T01:17:54.945064+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:17:54.945064+0000","last_undegraded":"2024-09-11T01:17:54.945064+0000","last_fullsized":"2024-09-11T01:17:54.945064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9062,"log_dups_size":0,"ondisk_log_size":9062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37117952,"num_objects":9062,"num_object_clones":0,"num_object_copies":18124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9062,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9062,"num_write_kb":36248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9047","reported_seq":9163,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.927856+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:55.927856+0000","last_peered":"2024-09-11T01:17:55.927856+0000","last_clean":"2024-09-11T01:17:55.927856+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:55.927856+0000","last_undegraded":"2024-09-11T01:17:55.927856+0000","last_fullsized":"2024-09-11T01:17:55.927856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37028323,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9056","reported_seq":9185,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.866506+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:55.866506+0000","last_peered":"2024-09-11T01:17:55.866506+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:55.866506+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9056,"log_dups_size":0,"ondisk_log_size":9056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37073379,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4937},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9334","reported_seq":9462,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.818655+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:55.818655+0000","last_peered":"2024-09-11T01:17:55.818655+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:55.818655+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9334,"log_dups_size":0,"ondisk_log_size":9334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38208454,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6087,"num_write_kb":24348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5047},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9098","reported_seq":9212,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.889731+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:55.889731+0000","last_peered":"2024-09-11T01:17:55.889731+0000","last_clean":"2024-09-11T01:17:55.889731+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:55.889731+0000","last_undegraded":"2024-09-11T01:17:55.889731+0000","last_fullsized":"2024-09-11T01:17:55.889731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6051,"num_write_kb":24204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'8987","reported_seq":17325,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:53.436967+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:53.436967+0000","last_peered":"2024-09-11T01:17:53.436967+0000","last_clean":"2024-09-11T01:17:53.436967+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:53.436967+0000","last_undegraded":"2024-09-11T01:17:53.436967+0000","last_fullsized":"2024-09-11T01:17:53.436967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":8987,"log_dups_size":0,"ondisk_log_size":8987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782563,"num_objects":8981,"num_object_clones":0,"num_object_copies":17962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8981,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6854,"num_write_kb":27416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9177","reported_seq":9301,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.872176+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:55.872176+0000","last_peered":"2024-09-11T01:17:55.872176+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:55.872176+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9177,"log_dups_size":0,"ondisk_log_size":9177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5541,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5064},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'8931","reported_seq":9020,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.144393+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:17:55.144393+0000","last_peered":"2024-09-11T01:17:55.144393+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:17:55.144393+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":8931,"log_dups_size":0,"ondisk_log_size":8931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36561379,"num_objects":8927,"num_object_clones":0,"num_object_copies":17854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6700,"num_write_kb":26800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":4900}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9203","reported_seq":9291,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.054955+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:17:55.054955+0000","last_peered":"2024-09-11T01:17:55.054955+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:17:55.054955+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9203,"log_dups_size":0,"ondisk_log_size":9203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37670912,"num_objects":9197,"num_object_clones":0,"num_object_copies":18394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6840,"num_write_kb":27360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5020}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9119","reported_seq":9829,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:55.821701+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:55.821701+0000","last_peered":"2024-09-11T01:17:55.821701+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:55.821701+0000","last_undegraded":"2024-09-11T01:17:55.821701+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9119,"log_dups_size":0,"ondisk_log_size":9119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5809,"num_write_kb":23236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":3872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9289","reported_seq":9418,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.934120+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:55.934120+0000","last_peered":"2024-09-11T01:17:55.934120+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:55.934120+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9289,"log_dups_size":0,"ondisk_log_size":9289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5089},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9197","reported_seq":9325,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.881289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:55.881289+0000","last_peered":"2024-09-11T01:17:55.881289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:55.881289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9197,"log_dups_size":0,"ondisk_log_size":9197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37654528,"num_objects":9193,"num_object_clones":0,"num_object_copies":18386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5045},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9023","reported_seq":9153,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.791416+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:55.791416+0000","last_peered":"2024-09-11T01:17:55.791416+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:55.791416+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9023,"log_dups_size":0,"ondisk_log_size":9023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5680,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4859},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595993310,"num_objects":145515,"num_object_clones":0,"num_object_copies":291030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":145515,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":104650,"num_write_kb":418600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18088419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1207541760,"data_stored":1207390014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145587,"ondisk_log_size":145587,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679236,"kb_used_data":422644,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93692604,"statfs":{"total":96636764160,"available":95941226496,"internally_reserved":0,"allocated":432787456,"data_stored":432520151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789984,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468652,"kb_used_data":252316,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93903188,"statfs":{"total":96636764160,"available":96156864512,"internally_reserved":0,"allocated":258371584,"data_stored":258056846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809356,"kb_used_data":502844,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93562484,"statfs":{"total":96636764160,"available":95807983616,"internally_reserved":0,"allocated":514912256,"data_stored":514640874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":515584000,"data_stored":515547870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259248128,"data_stored":259168642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432709632,"data_stored":432673502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:58.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 202 20 146762 146742 2.83706 1.85938 0.0317195 0.0275298 2024-09-11T01:17:59.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 203 20 147236 147216 2.83221 1.85156 0.0597775 0.0275778 2024-09-11T01:18:00.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 204 20 147713 147693 2.82746 1.86328 0.02398 0.0276238 2024-09-11T01:18:00.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:00.995 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:00.998 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:01.021 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-11T01:18:00.480655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599442161,"num_objects":146359,"num_object_clones":0,"num_object_copies":292718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":146359,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":105494,"num_write_kb":421970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4419,"num_bytes_recovered":18088438,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146432,"ondisk_log_size":146432,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1977028,"kb_used_data":1189268,"kb_used_omap":12,"kb_used_meta":787699,"kb_avail":281138492,"statfs":{"total":289910292480,"available":287885815808,"internally_reserved":0,"allocated":1217810432,"data_stored":1216957007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":806603783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,7,12,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-138,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":565248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003636"},"pg_stats":[{"pgid":"2.f","version":"41'9009","reported_seq":9075,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.896755+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:55.896755+0000","last_peered":"2024-09-11T01:17:55.896755+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:55.896755+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9009,"num_write_kb":36036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9255","reported_seq":9330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.013285+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:00.013285+0000","last_peered":"2024-09-11T01:18:00.013285+0000","last_clean":"2024-09-11T01:18:00.013285+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:00.013285+0000","last_undegraded":"2024-09-11T01:18:00.013285+0000","last_fullsized":"2024-09-11T01:18:00.013285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9255,"log_dups_size":0,"ondisk_log_size":9255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884870,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7353,"num_write_kb":29412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9061","reported_seq":9139,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.134133+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:00.134133+0000","last_peered":"2024-09-11T01:18:00.134133+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:00.134133+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9219","reported_seq":9259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.152306+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:00.152306+0000","last_peered":"2024-09-11T01:18:00.152306+0000","last_clean":"2024-09-11T01:18:00.152306+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:00.152306+0000","last_undegraded":"2024-09-11T01:18:00.152306+0000","last_fullsized":"2024-09-11T01:18:00.152306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9219,"log_dups_size":0,"ondisk_log_size":9219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37761024,"num_objects":9219,"num_object_clones":0,"num_object_copies":18438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":36876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9047","reported_seq":9163,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.927856+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:55.927856+0000","last_peered":"2024-09-11T01:17:55.927856+0000","last_clean":"2024-09-11T01:17:55.927856+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:55.927856+0000","last_undegraded":"2024-09-11T01:17:55.927856+0000","last_fullsized":"2024-09-11T01:17:55.927856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37028323,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9056","reported_seq":9185,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.866506+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:55.866506+0000","last_peered":"2024-09-11T01:17:55.866506+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:55.866506+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9056,"log_dups_size":0,"ondisk_log_size":9056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37073379,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4937},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9334","reported_seq":9462,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.818655+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:55.818655+0000","last_peered":"2024-09-11T01:17:55.818655+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:55.818655+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9334,"log_dups_size":0,"ondisk_log_size":9334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38208454,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6087,"num_write_kb":24348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5047},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9098","reported_seq":9212,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.889731+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:55.889731+0000","last_peered":"2024-09-11T01:17:55.889731+0000","last_clean":"2024-09-11T01:17:55.889731+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:55.889731+0000","last_undegraded":"2024-09-11T01:17:55.889731+0000","last_fullsized":"2024-09-11T01:17:55.889731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6051,"num_write_kb":24204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9129","reported_seq":17467,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.548699+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:58.548699+0000","last_peered":"2024-09-11T01:17:58.548699+0000","last_clean":"2024-09-11T01:17:58.548699+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:58.548699+0000","last_undegraded":"2024-09-11T01:17:58.548699+0000","last_fullsized":"2024-09-11T01:17:58.548699+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9129,"log_dups_size":0,"ondisk_log_size":9129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37364195,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6996,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9177","reported_seq":9301,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.872176+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:55.872176+0000","last_peered":"2024-09-11T01:17:55.872176+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:55.872176+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9177,"log_dups_size":0,"ondisk_log_size":9177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5541,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5064},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9068","reported_seq":9157,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.062399+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:00.062399+0000","last_peered":"2024-09-11T01:18:00.062399+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:00.062399+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122531,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9348","reported_seq":9436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.152677+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:00.152677+0000","last_peered":"2024-09-11T01:18:00.152677+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:00.152677+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9348,"log_dups_size":0,"ondisk_log_size":9348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5165}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9119","reported_seq":9829,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:55.821701+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:55.821701+0000","last_peered":"2024-09-11T01:17:55.821701+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:55.821701+0000","last_undegraded":"2024-09-11T01:17:55.821701+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9119,"log_dups_size":0,"ondisk_log_size":9119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5809,"num_write_kb":23236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":3872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9289","reported_seq":9418,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.934120+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:55.934120+0000","last_peered":"2024-09-11T01:17:55.934120+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:55.934120+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9289,"log_dups_size":0,"ondisk_log_size":9289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5089},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9197","reported_seq":9325,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.881289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:55.881289+0000","last_peered":"2024-09-11T01:17:55.881289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:55.881289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9197,"log_dups_size":0,"ondisk_log_size":9197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37654528,"num_objects":9193,"num_object_clones":0,"num_object_copies":18386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5045},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9023","reported_seq":9153,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.791416+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:55.791416+0000","last_peered":"2024-09-11T01:17:55.791416+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:55.791416+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9023,"log_dups_size":0,"ondisk_log_size":9023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5680,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4859},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":599442142,"num_objects":146357,"num_object_clones":0,"num_object_copies":292714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":146357,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":105492,"num_write_kb":421968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18088419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1219432448,"data_stored":1219280702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146429,"ondisk_log_size":146429,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687940,"kb_used_data":427188,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93683900,"statfs":{"total":96636764160,"available":95932313600,"internally_reserved":0,"allocated":437440512,"data_stored":437173207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":266989230},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789985,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479732,"kb_used_data":259236,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93892108,"statfs":{"total":96636764160,"available":96145518592,"internally_reserved":0,"allocated":265457664,"data_stored":265142926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809356,"kb_used_data":502844,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93562484,"statfs":{"total":96636764160,"available":95807983616,"internally_reserved":0,"allocated":514912256,"data_stored":514640874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":515584000,"data_stored":515547870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266473472,"data_stored":266393986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437374976,"data_stored":437338846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:01.022 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:01.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 205 20 148139 148119 2.82178 1.66406 0.0419276 0.0276793 2024-09-11T01:18:01.615 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:01.615 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:01.637 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-11T01:18:00.480655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599442161,"num_objects":146359,"num_object_clones":0,"num_object_copies":292718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":146359,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":105494,"num_write_kb":421970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4419,"num_bytes_recovered":18088438,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146432,"ondisk_log_size":146432,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1977028,"kb_used_data":1189268,"kb_used_omap":12,"kb_used_meta":787699,"kb_avail":281138492,"statfs":{"total":289910292480,"available":287885815808,"internally_reserved":0,"allocated":1217810432,"data_stored":1216957007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":806603783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,7,12,15,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-138,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":138,"num_bytes_recovered":565248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003636"},"pg_stats":[{"pgid":"2.f","version":"41'9009","reported_seq":9075,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.896755+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:17:55.896755+0000","last_peered":"2024-09-11T01:17:55.896755+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:17:55.896755+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9009,"num_write_kb":36036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4886},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9255","reported_seq":9330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.013285+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:00.013285+0000","last_peered":"2024-09-11T01:18:00.013285+0000","last_clean":"2024-09-11T01:18:00.013285+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:00.013285+0000","last_undegraded":"2024-09-11T01:18:00.013285+0000","last_fullsized":"2024-09-11T01:18:00.013285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9255,"log_dups_size":0,"ondisk_log_size":9255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884870,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7353,"num_write_kb":29412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9061","reported_seq":9139,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.134133+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:00.134133+0000","last_peered":"2024-09-11T01:18:00.134133+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:00.134133+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9219","reported_seq":9259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.152306+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:00.152306+0000","last_peered":"2024-09-11T01:18:00.152306+0000","last_clean":"2024-09-11T01:18:00.152306+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:00.152306+0000","last_undegraded":"2024-09-11T01:18:00.152306+0000","last_fullsized":"2024-09-11T01:18:00.152306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9219,"log_dups_size":0,"ondisk_log_size":9219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37761024,"num_objects":9219,"num_object_clones":0,"num_object_copies":18438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":36876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9047","reported_seq":9163,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.927856+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:17:55.927856+0000","last_peered":"2024-09-11T01:17:55.927856+0000","last_clean":"2024-09-11T01:17:55.927856+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:17:55.927856+0000","last_undegraded":"2024-09-11T01:17:55.927856+0000","last_fullsized":"2024-09-11T01:17:55.927856+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37028323,"num_objects":9041,"num_object_clones":0,"num_object_copies":18082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5845,"num_write_kb":23380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9056","reported_seq":9185,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.866506+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:17:55.866506+0000","last_peered":"2024-09-11T01:17:55.866506+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:17:55.866506+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9056,"log_dups_size":0,"ondisk_log_size":9056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37073379,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5544,"num_write_kb":22176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4937},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9334","reported_seq":9462,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.818655+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:17:55.818655+0000","last_peered":"2024-09-11T01:17:55.818655+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:17:55.818655+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9334,"log_dups_size":0,"ondisk_log_size":9334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38208454,"num_objects":9330,"num_object_clones":0,"num_object_copies":18660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9330,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6087,"num_write_kb":24348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5047},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9098","reported_seq":9212,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:55.889731+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:17:55.889731+0000","last_peered":"2024-09-11T01:17:55.889731+0000","last_clean":"2024-09-11T01:17:55.889731+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:17:55.889731+0000","last_undegraded":"2024-09-11T01:17:55.889731+0000","last_fullsized":"2024-09-11T01:17:55.889731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9098,"log_dups_size":0,"ondisk_log_size":9098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6051,"num_write_kb":24204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9129","reported_seq":17467,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.548699+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:58.548699+0000","last_peered":"2024-09-11T01:17:58.548699+0000","last_clean":"2024-09-11T01:17:58.548699+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:58.548699+0000","last_undegraded":"2024-09-11T01:17:58.548699+0000","last_fullsized":"2024-09-11T01:17:58.548699+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9129,"log_dups_size":0,"ondisk_log_size":9129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37364195,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6996,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9177","reported_seq":9301,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.872176+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:17:55.872176+0000","last_peered":"2024-09-11T01:17:55.872176+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:17:55.872176+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9177,"log_dups_size":0,"ondisk_log_size":9177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5541,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5064},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9068","reported_seq":9157,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.062399+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:00.062399+0000","last_peered":"2024-09-11T01:18:00.062399+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:00.062399+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122531,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9348","reported_seq":9436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.152677+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:00.152677+0000","last_peered":"2024-09-11T01:18:00.152677+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:00.152677+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9348,"log_dups_size":0,"ondisk_log_size":9348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5165}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9119","reported_seq":9829,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:55.821701+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:17:55.821701+0000","last_peered":"2024-09-11T01:17:55.821701+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:17:55.821701+0000","last_undegraded":"2024-09-11T01:17:55.821701+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9119,"log_dups_size":0,"ondisk_log_size":9119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37343232,"num_objects":9117,"num_object_clones":0,"num_object_copies":18234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":9117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5809,"num_write_kb":23236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":3872}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9289","reported_seq":9418,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.934120+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:17:55.934120+0000","last_peered":"2024-09-11T01:17:55.934120+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:17:55.934120+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9289,"log_dups_size":0,"ondisk_log_size":9289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38014976,"num_objects":9281,"num_object_clones":0,"num_object_copies":18562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5089},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9197","reported_seq":9325,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.881289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:17:55.881289+0000","last_peered":"2024-09-11T01:17:55.881289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:17:55.881289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9197,"log_dups_size":0,"ondisk_log_size":9197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37654528,"num_objects":9193,"num_object_clones":0,"num_object_copies":18386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5045},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9023","reported_seq":9153,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:55.791416+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:17:55.791416+0000","last_peered":"2024-09-11T01:17:55.791416+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:17:55.791416+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9023,"log_dups_size":0,"ondisk_log_size":9023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36910022,"num_objects":9013,"num_object_clones":0,"num_object_copies":18026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9013,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5680,"num_write_kb":22720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4859},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":599442142,"num_objects":146357,"num_object_clones":0,"num_object_copies":292714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3872,"num_objects_unfound":0,"num_objects_dirty":146357,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":105492,"num_write_kb":421968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4418,"num_bytes_recovered":18088419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1219432448,"data_stored":1219280702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146429,"ondisk_log_size":146429,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687940,"kb_used_data":427188,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93683900,"statfs":{"total":96636764160,"available":95932313600,"internally_reserved":0,"allocated":437440512,"data_stored":437173207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":266989230},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789985,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479732,"kb_used_data":259236,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93892108,"statfs":{"total":96636764160,"available":96145518592,"internally_reserved":0,"allocated":265457664,"data_stored":265142926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809356,"kb_used_data":502844,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93562484,"statfs":{"total":96636764160,"available":95807983616,"internally_reserved":0,"allocated":514912256,"data_stored":514640874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,7,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":515584000,"data_stored":515547870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266473472,"data_stored":266393986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437374976,"data_stored":437338846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:01.638 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:01.638 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'9009", 'reported_seq': 9075, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.896755+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:17:55.896755+0000', 'last_peered': '2024-09-11T01:17:55.896755+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:17:55.896755+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 9009, 'log_dups_size': 0, 'ondisk_log_size': 9009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36900864, 'num_objects': 9009, 'num_object_clones': 0, 'num_object_copies': 18018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9009, 'num_write_kb': 36036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4886}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9061", 'reported_seq': 9139, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:00.134133+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:00.134133+0000', 'last_peered': '2024-09-11T01:18:00.134133+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:00.134133+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9061, 'log_dups_size': 0, 'ondisk_log_size': 9061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37097472, 'num_objects': 9057, 'num_object_clones': 0, 'num_object_copies': 18114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7122, 'num_write_kb': 28488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5052}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9056", 'reported_seq': 9185, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.866506+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:17:55.866506+0000', 'last_peered': '2024-09-11T01:17:55.866506+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:17:55.866506+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9056, 'log_dups_size': 0, 'ondisk_log_size': 9056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37073379, 'num_objects': 9052, 'num_object_clones': 0, 'num_object_copies': 18104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9052, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5544, 'num_write_kb': 22176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4937}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9334", 'reported_seq': 9462, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.818655+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:17:55.818655+0000', 'last_peered': '2024-09-11T01:17:55.818655+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:17:55.818655+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9334, 'log_dups_size': 0, 'ondisk_log_size': 9334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38208454, 'num_objects': 9330, 'num_object_clones': 0, 'num_object_copies': 18660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9330, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6087, 'num_write_kb': 24348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5047}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'9177", 'reported_seq': 9301, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.872176+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:17:55.872176+0000', 'last_peered': '2024-09-11T01:17:55.872176+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:17:55.872176+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 9177, 'log_dups_size': 0, 'ondisk_log_size': 9177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37556224, 'num_objects': 9169, 'num_object_clones': 0, 'num_object_copies': 18338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5541, 'num_write_kb': 22164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5064}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:01.639 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9068", 'reported_seq': 9157, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:00.062399+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:00.062399+0000', 'last_peered': '2024-09-11T01:18:00.062399+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:00.062399+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9068, 'log_dups_size': 0, 'ondisk_log_size': 9068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37122531, 'num_objects': 9064, 'num_object_clones': 0, 'num_object_copies': 18128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9064, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6837, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5037}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'9348", 'reported_seq': 9436, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:00.152677+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:00.152677+0000', 'last_peered': '2024-09-11T01:18:00.152677+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:00.152677+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 9348, 'log_dups_size': 0, 'ondisk_log_size': 9348, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38264832, 'num_objects': 9342, 'num_object_clones': 0, 'num_object_copies': 18684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6985, 'num_write_kb': 27940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5165}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'9119", 'reported_seq': 9829, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:55.821701+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:17:55.821701+0000', 'last_peered': '2024-09-11T01:17:55.821701+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:17:55.821701+0000', 'last_undegraded': '2024-09-11T01:17:55.821701+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 9119, 'log_dups_size': 0, 'ondisk_log_size': 9119, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37343232, 'num_objects': 9117, 'num_object_clones': 0, 'num_object_copies': 18234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3872, 'num_objects_unfound': 0, 'num_objects_dirty': 9117, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5809, 'num_write_kb': 23236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 1196032, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5245}, {'shards': '0,2', 'objects': 3872}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9289", 'reported_seq': 9418, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.934120+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:17:55.934120+0000', 'last_peered': '2024-09-11T01:17:55.934120+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:17:55.934120+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9289, 'log_dups_size': 0, 'ondisk_log_size': 9289, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38014976, 'num_objects': 9281, 'num_object_clones': 0, 'num_object_copies': 18562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9281, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5682, 'num_write_kb': 22728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5089}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'9197", 'reported_seq': 9325, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.881289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:17:55.881289+0000', 'last_peered': '2024-09-11T01:17:55.881289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:17:55.881289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 9197, 'log_dups_size': 0, 'ondisk_log_size': 9197, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37654528, 'num_objects': 9193, 'num_object_clones': 0, 'num_object_copies': 18386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5732, 'num_write_kb': 22928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5045}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:01.640 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9023", 'reported_seq': 9153, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:55.791416+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:17:55.791416+0000', 'last_peered': '2024-09-11T01:17:55.791416+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:17:55.791416+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9023, 'log_dups_size': 0, 'ondisk_log_size': 9023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36910022, 'num_objects': 9013, 'num_object_clones': 0, 'num_object_copies': 18026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9013, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5680, 'num_write_kb': 22720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4859}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:01.641 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:02.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:02.340 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/292718 objects degraded (14.124%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":282,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146359,"data_bytes":599442161,"bytes_used":2024476672,"bytes_avail":287885815808,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":292718,"degraded_ratio":0.14123832494072794,"misplaced_objects":3872,"misplaced_total":292718,"misplaced_ratio":0.013227748208173053,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":47089,"recovering_keys_per_sec":0,"num_objects_recovered":138,"num_bytes_recovered":565248,"num_keys_recovered":0,"write_bytes_sec":1803058,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:02.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 206 20 148555 148535 2.81597 1.625 0.0117024 0.0277343 2024-09-11T01:18:02.368 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:02.369 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:02.966 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:02.968 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:02.997 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-11T01:18:02.481144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605328113,"num_objects":147796,"num_object_clones":0,"num_object_copies":295592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3809,"num_objects_unfound":0,"num_objects_dirty":147796,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":106931,"num_write_kb":427718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4482,"num_bytes_recovered":18346486,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147869,"ondisk_log_size":147869,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1988104,"kb_used_data":1196184,"kb_used_omap":12,"kb_used_meta":791859,"kb_avail":281127416,"statfs":{"total":289910292480,"available":287874473984,"internally_reserved":0,"allocated":1224892416,"data_stored":1224038991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":810863623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,11,5,6,9,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24457216,"num_objects":5971,"num_object_clones":0,"num_object_copies":11942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-201,"num_objects_unfound":0,"num_objects_dirty":5971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5971,"num_write_kb":23884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003654"},"pg_stats":[{"pgid":"2.f","version":"41'9163","reported_seq":9227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.932630+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:00.932630+0000","last_peered":"2024-09-11T01:18:00.932630+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:00.932630+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9163,"log_dups_size":0,"ondisk_log_size":9163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37531648,"num_objects":9163,"num_object_clones":0,"num_object_copies":18326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9163,"num_write_kb":36652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5040},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9255","reported_seq":9330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.013285+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:00.013285+0000","last_peered":"2024-09-11T01:18:00.013285+0000","last_clean":"2024-09-11T01:18:00.013285+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:00.013285+0000","last_undegraded":"2024-09-11T01:18:00.013285+0000","last_fullsized":"2024-09-11T01:18:00.013285+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9255,"log_dups_size":0,"ondisk_log_size":9255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884870,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7353,"num_write_kb":29412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9061","reported_seq":9139,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.134133+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:00.134133+0000","last_peered":"2024-09-11T01:18:00.134133+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:00.134133+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9061,"log_dups_size":0,"ondisk_log_size":9061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097472,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7122,"num_write_kb":28488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5052}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9219","reported_seq":9259,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.152306+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:00.152306+0000","last_peered":"2024-09-11T01:18:00.152306+0000","last_clean":"2024-09-11T01:18:00.152306+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:00.152306+0000","last_undegraded":"2024-09-11T01:18:00.152306+0000","last_fullsized":"2024-09-11T01:18:00.152306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9219,"log_dups_size":0,"ondisk_log_size":9219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37761024,"num_objects":9219,"num_object_clones":0,"num_object_copies":18438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":36876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9188","reported_seq":9304,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.916050+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:00.916050+0000","last_peered":"2024-09-11T01:18:00.916050+0000","last_clean":"2024-09-11T01:18:00.916050+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:00.916050+0000","last_undegraded":"2024-09-11T01:18:00.916050+0000","last_fullsized":"2024-09-11T01:18:00.916050+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9188,"log_dups_size":0,"ondisk_log_size":9188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37605859,"num_objects":9182,"num_object_clones":0,"num_object_copies":18364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9182,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5986,"num_write_kb":23944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9195","reported_seq":9324,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.860549+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:00.860549+0000","last_peered":"2024-09-11T01:18:00.860549+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:00.860549+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37642723,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5683,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5076},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9478","reported_seq":9606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.967320+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:00.967320+0000","last_peered":"2024-09-11T01:18:00.967320+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:00.967320+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9478,"log_dups_size":0,"ondisk_log_size":9478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38798278,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6231,"num_write_kb":24924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5191},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9253","reported_seq":9367,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.832227+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:00.832227+0000","last_peered":"2024-09-11T01:18:00.832227+0000","last_clean":"2024-09-11T01:18:00.832227+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:00.832227+0000","last_undegraded":"2024-09-11T01:18:00.832227+0000","last_fullsized":"2024-09-11T01:18:00.832227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9253,"log_dups_size":0,"ondisk_log_size":9253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37892096,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6206,"num_write_kb":24824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9129","reported_seq":17467,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.548699+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:17:58.548699+0000","last_peered":"2024-09-11T01:17:58.548699+0000","last_clean":"2024-09-11T01:17:58.548699+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:17:58.548699+0000","last_undegraded":"2024-09-11T01:17:58.548699+0000","last_fullsized":"2024-09-11T01:17:58.548699+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9129,"log_dups_size":0,"ondisk_log_size":9129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37364195,"num_objects":9123,"num_object_clones":0,"num_object_copies":18246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6996,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9335","reported_seq":9459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.632721+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:00.632721+0000","last_peered":"2024-09-11T01:18:00.632721+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:00.632721+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38203392,"num_objects":9327,"num_object_clones":0,"num_object_copies":18654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5699,"num_write_kb":22796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5222},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9068","reported_seq":9157,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.062399+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:00.062399+0000","last_peered":"2024-09-11T01:18:00.062399+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:00.062399+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9068,"log_dups_size":0,"ondisk_log_size":9068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37122531,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9348","reported_seq":9436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.152677+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:00.152677+0000","last_peered":"2024-09-11T01:18:00.152677+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:00.152677+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9348,"log_dups_size":0,"ondisk_log_size":9348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38264832,"num_objects":9342,"num_object_clones":0,"num_object_copies":18684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5165}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9269","reported_seq":10103,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:00.941409+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:00.941409+0000","last_peered":"2024-09-11T01:18:00.941409+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:00.941409+0000","last_undegraded":"2024-09-11T01:18:00.941409+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9269,"log_dups_size":0,"ondisk_log_size":9269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3809,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5959,"num_write_kb":23836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5458},{"shards":"0,2","objects":3809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9423","reported_seq":9552,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.846474+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:00.846474+0000","last_peered":"2024-09-11T01:18:00.846474+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:00.846474+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9423,"log_dups_size":0,"ondisk_log_size":9423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38563840,"num_objects":9415,"num_object_clones":0,"num_object_copies":18830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5816,"num_write_kb":23264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9324","reported_seq":9452,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.822277+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:00.822277+0000","last_peered":"2024-09-11T01:18:00.822277+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:00.822277+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9324,"log_dups_size":0,"ondisk_log_size":9324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38174720,"num_objects":9320,"num_object_clones":0,"num_object_copies":18640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5172},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9158","reported_seq":9288,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:00.952147+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:00.952147+0000","last_peered":"2024-09-11T01:18:00.952147+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:00.952147+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9158,"log_dups_size":0,"ondisk_log_size":9158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37462982,"num_objects":9148,"num_object_clones":0,"num_object_copies":18296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5815,"num_write_kb":23260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4994},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605328094,"num_objects":147794,"num_object_clones":0,"num_object_copies":295588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3809,"num_objects_unfound":0,"num_objects_dirty":147794,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":106929,"num_write_kb":427716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4481,"num_bytes_recovered":18346467,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226477568,"data_stored":1226325822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147866,"ondisk_log_size":147866,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":687940,"kb_used_data":427188,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93683900,"statfs":{"total":96636764160,"available":95932313600,"internally_reserved":0,"allocated":437440512,"data_stored":437173207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":266989230},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789985,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479732,"kb_used_data":259236,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93892108,"statfs":{"total":96636764160,"available":96145518592,"internally_reserved":0,"allocated":265457664,"data_stored":265142926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820432,"kb_used_data":509760,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93551408,"statfs":{"total":96636764160,"available":95796641792,"internally_reserved":0,"allocated":521994240,"data_stored":521722858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,0,5,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":522629120,"data_stored":522592990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266473472,"data_stored":266393986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437374976,"data_stored":437338846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:03.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 207 20 148939 148919 2.80961 1.5 0.00776289 0.0277826 2024-09-11T01:18:04.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 208 20 149377 149357 2.80433 1.71094 0.0328599 0.0278521 2024-09-11T01:18:05.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 209 20 149841 149821 2.79959 1.8125 0.0201646 0.0278972 2024-09-11T01:18:05.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:06.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 210 20 150287 150267 2.79455 1.74219 0.0349831 0.0279471 2024-09-11T01:18:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:06.583 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:06.604 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-11T01:18:06.482174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614191857,"num_objects":149960,"num_object_clones":0,"num_object_copies":299920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149960,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109095,"num_write_kb":436374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18686454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150033,"ondisk_log_size":150033,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1968392,"kb_used_data":1213912,"kb_used_omap":12,"kb_used_meta":754419,"kb_avail":281147128,"statfs":{"total":289910292480,"available":287894659072,"internally_reserved":0,"allocated":1243045888,"data_stored":1242192463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":772525063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,14,8,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6573,"num_write_kb":26292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003112"},"pg_stats":[{"pgid":"2.f","version":"41'9282","reported_seq":9350,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.914268+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:05.914268+0000","last_peered":"2024-09-11T01:18:05.914268+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:05.914268+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9282,"num_write_kb":37128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9409","reported_seq":9483,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.055815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:05.055815+0000","last_peered":"2024-09-11T01:18:05.055815+0000","last_clean":"2024-09-11T01:18:05.055815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:05.055815+0000","last_undegraded":"2024-09-11T01:18:05.055815+0000","last_fullsized":"2024-09-11T01:18:05.055815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9409,"log_dups_size":0,"ondisk_log_size":9409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515654,"num_objects":9405,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7507,"num_write_kb":30028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9195","reported_seq":9273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.120065+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:05.120065+0000","last_peered":"2024-09-11T01:18:05.120065+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:05.120065+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7256,"num_write_kb":29024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9359","reported_seq":9399,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.109015+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:05.109015+0000","last_peered":"2024-09-11T01:18:05.109015+0000","last_clean":"2024-09-11T01:18:05.109015+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:05.109015+0000","last_undegraded":"2024-09-11T01:18:05.109015+0000","last_fullsized":"2024-09-11T01:18:05.109015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9359,"log_dups_size":0,"ondisk_log_size":9359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334464,"num_objects":9359,"num_object_clones":0,"num_object_copies":18718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9359,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9320","reported_seq":9434,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.981913+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:05.981913+0000","last_peered":"2024-09-11T01:18:05.981913+0000","last_clean":"2024-09-11T01:18:05.981913+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:05.981913+0000","last_undegraded":"2024-09-11T01:18:05.981913+0000","last_fullsized":"2024-09-11T01:18:05.981913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146531,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6118,"num_write_kb":24472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9326","reported_seq":9454,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.978556+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:05.978556+0000","last_peered":"2024-09-11T01:18:05.978556+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:05.978556+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9326,"log_dups_size":0,"ondisk_log_size":9326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9606","reported_seq":9734,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.935803+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:05.935803+0000","last_peered":"2024-09-11T01:18:05.935803+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:05.935803+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322566,"num_objects":9602,"num_object_clones":0,"num_object_copies":19204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9602,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5319},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9389","reported_seq":9503,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.988909+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:05.988909+0000","last_peered":"2024-09-11T01:18:05.988909+0000","last_clean":"2024-09-11T01:18:05.988909+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:05.988909+0000","last_undegraded":"2024-09-11T01:18:05.988909+0000","last_fullsized":"2024-09-11T01:18:05.988909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38449152,"num_objects":9387,"num_object_clones":0,"num_object_copies":18774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9257","reported_seq":17595,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:03.622835+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:03.622835+0000","last_peered":"2024-09-11T01:18:03.622835+0000","last_clean":"2024-09-11T01:18:03.622835+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:03.622835+0000","last_undegraded":"2024-09-11T01:18:03.622835+0000","last_fullsized":"2024-09-11T01:18:03.622835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9257,"log_dups_size":0,"ondisk_log_size":9257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888483,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7124,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9470","reported_seq":9594,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.951638+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:05.951638+0000","last_peered":"2024-09-11T01:18:05.951638+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:05.951638+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756352,"num_objects":9462,"num_object_clones":0,"num_object_copies":18924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5834,"num_write_kb":23336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5357},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9221","reported_seq":9310,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.168117+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:05.168117+0000","last_peered":"2024-09-11T01:18:05.168117+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:05.168117+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37749219,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6990,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9473","reported_seq":9561,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.151550+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:05.151550+0000","last_peered":"2024-09-11T01:18:05.151550+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:05.151550+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9473,"log_dups_size":0,"ondisk_log_size":9473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7110,"num_write_kb":28440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5290}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9408","reported_seq":10410,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:06.015645+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:06.015645+0000","last_peered":"2024-09-11T01:18:06.015645+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:06.015645+0000","last_undegraded":"2024-09-11T01:18:06.015645+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9408,"log_dups_size":0,"ondisk_log_size":9408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6098,"num_write_kb":24392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":438,"num_bytes_recovered":1794048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5680},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9552","reported_seq":9681,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.981044+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:05.981044+0000","last_peered":"2024-09-11T01:18:05.981044+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:05.981044+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5945,"num_write_kb":23780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9478","reported_seq":9606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.885609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:05.885609+0000","last_peered":"2024-09-11T01:18:05.885609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:05.885609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9478,"log_dups_size":0,"ondisk_log_size":9478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9285","reported_seq":9415,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.933756+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:05.933756+0000","last_peered":"2024-09-11T01:18:05.933756+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:05.933756+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37983174,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5121},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":614191838,"num_objects":149958,"num_object_clones":0,"num_object_copies":299916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149958,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109093,"num_write_kb":436372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4564,"num_bytes_recovered":18686435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1244471296,"data_stored":1244319550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150030,"ondisk_log_size":150030,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679852,"kb_used_data":431580,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93691988,"statfs":{"total":96636764160,"available":95940595712,"internally_reserved":0,"allocated":441937920,"data_stored":441670615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789986,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474276,"kb_used_data":266260,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93897564,"statfs":{"total":96636764160,"available":96151105536,"internally_reserved":0,"allocated":272650240,"data_stored":272335502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814264,"kb_used_data":516072,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93557576,"statfs":{"total":96636764160,"available":95802957824,"internally_reserved":0,"allocated":528457728,"data_stored":528186346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,7,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529305600,"data_stored":529269470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273342464,"data_stored":273262978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441823232,"data_stored":441787102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:06.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:07.172 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:07.174 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:07.198 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-11T01:18:06.482174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614191857,"num_objects":149960,"num_object_clones":0,"num_object_copies":299920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149960,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109095,"num_write_kb":436374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18686454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150033,"ondisk_log_size":150033,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1968392,"kb_used_data":1213912,"kb_used_omap":12,"kb_used_meta":754419,"kb_avail":281147128,"statfs":{"total":289910292480,"available":287894659072,"internally_reserved":0,"allocated":1243045888,"data_stored":1242192463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":772525063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,14,8,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26923008,"num_objects":6573,"num_object_clones":0,"num_object_copies":13146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":6573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6573,"num_write_kb":26292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003112"},"pg_stats":[{"pgid":"2.f","version":"41'9282","reported_seq":9350,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.914268+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:05.914268+0000","last_peered":"2024-09-11T01:18:05.914268+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:05.914268+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9282,"num_write_kb":37128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9409","reported_seq":9483,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.055815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:05.055815+0000","last_peered":"2024-09-11T01:18:05.055815+0000","last_clean":"2024-09-11T01:18:05.055815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:05.055815+0000","last_undegraded":"2024-09-11T01:18:05.055815+0000","last_fullsized":"2024-09-11T01:18:05.055815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9409,"log_dups_size":0,"ondisk_log_size":9409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515654,"num_objects":9405,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7507,"num_write_kb":30028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9195","reported_seq":9273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.120065+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:05.120065+0000","last_peered":"2024-09-11T01:18:05.120065+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:05.120065+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7256,"num_write_kb":29024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9359","reported_seq":9399,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.109015+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:05.109015+0000","last_peered":"2024-09-11T01:18:05.109015+0000","last_clean":"2024-09-11T01:18:05.109015+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:05.109015+0000","last_undegraded":"2024-09-11T01:18:05.109015+0000","last_fullsized":"2024-09-11T01:18:05.109015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9359,"log_dups_size":0,"ondisk_log_size":9359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334464,"num_objects":9359,"num_object_clones":0,"num_object_copies":18718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9359,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9320","reported_seq":9434,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.981913+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:05.981913+0000","last_peered":"2024-09-11T01:18:05.981913+0000","last_clean":"2024-09-11T01:18:05.981913+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:05.981913+0000","last_undegraded":"2024-09-11T01:18:05.981913+0000","last_fullsized":"2024-09-11T01:18:05.981913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146531,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6118,"num_write_kb":24472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9326","reported_seq":9454,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.978556+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:05.978556+0000","last_peered":"2024-09-11T01:18:05.978556+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:05.978556+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9326,"log_dups_size":0,"ondisk_log_size":9326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9606","reported_seq":9734,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.935803+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:05.935803+0000","last_peered":"2024-09-11T01:18:05.935803+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:05.935803+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322566,"num_objects":9602,"num_object_clones":0,"num_object_copies":19204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9602,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5319},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9389","reported_seq":9503,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.988909+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:05.988909+0000","last_peered":"2024-09-11T01:18:05.988909+0000","last_clean":"2024-09-11T01:18:05.988909+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:05.988909+0000","last_undegraded":"2024-09-11T01:18:05.988909+0000","last_fullsized":"2024-09-11T01:18:05.988909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38449152,"num_objects":9387,"num_object_clones":0,"num_object_copies":18774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9257","reported_seq":17595,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:03.622835+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:03.622835+0000","last_peered":"2024-09-11T01:18:03.622835+0000","last_clean":"2024-09-11T01:18:03.622835+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:03.622835+0000","last_undegraded":"2024-09-11T01:18:03.622835+0000","last_fullsized":"2024-09-11T01:18:03.622835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9257,"log_dups_size":0,"ondisk_log_size":9257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888483,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7124,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9470","reported_seq":9594,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.951638+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:05.951638+0000","last_peered":"2024-09-11T01:18:05.951638+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:05.951638+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756352,"num_objects":9462,"num_object_clones":0,"num_object_copies":18924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5834,"num_write_kb":23336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5357},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9221","reported_seq":9310,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.168117+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:05.168117+0000","last_peered":"2024-09-11T01:18:05.168117+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:05.168117+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37749219,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6990,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9473","reported_seq":9561,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.151550+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:05.151550+0000","last_peered":"2024-09-11T01:18:05.151550+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:05.151550+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9473,"log_dups_size":0,"ondisk_log_size":9473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7110,"num_write_kb":28440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5290}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9408","reported_seq":10410,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:06.015645+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:06.015645+0000","last_peered":"2024-09-11T01:18:06.015645+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:06.015645+0000","last_undegraded":"2024-09-11T01:18:06.015645+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9408,"log_dups_size":0,"ondisk_log_size":9408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6098,"num_write_kb":24392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":438,"num_bytes_recovered":1794048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5680},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9552","reported_seq":9681,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.981044+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:05.981044+0000","last_peered":"2024-09-11T01:18:05.981044+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:05.981044+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5945,"num_write_kb":23780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9478","reported_seq":9606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.885609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:05.885609+0000","last_peered":"2024-09-11T01:18:05.885609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:05.885609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9478,"log_dups_size":0,"ondisk_log_size":9478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9285","reported_seq":9415,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.933756+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:05.933756+0000","last_peered":"2024-09-11T01:18:05.933756+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:05.933756+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37983174,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5121},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":614191838,"num_objects":149958,"num_object_clones":0,"num_object_copies":299916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149958,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109093,"num_write_kb":436372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4564,"num_bytes_recovered":18686435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1244471296,"data_stored":1244319550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150030,"ondisk_log_size":150030,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679852,"kb_used_data":431580,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93691988,"statfs":{"total":96636764160,"available":95940595712,"internally_reserved":0,"allocated":441937920,"data_stored":441670615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789986,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474276,"kb_used_data":266260,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93897564,"statfs":{"total":96636764160,"available":96151105536,"internally_reserved":0,"allocated":272650240,"data_stored":272335502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814264,"kb_used_data":516072,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93557576,"statfs":{"total":96636764160,"available":95802957824,"internally_reserved":0,"allocated":528457728,"data_stored":528186346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,7,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529305600,"data_stored":529269470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273342464,"data_stored":273262978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441823232,"data_stored":441787102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'9282", 'reported_seq': 9350, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.914268+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:05.914268+0000', 'last_peered': '2024-09-11T01:18:05.914268+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:05.914268+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 9282, 'log_dups_size': 0, 'ondisk_log_size': 9282, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38019072, 'num_objects': 9282, 'num_object_clones': 0, 'num_object_copies': 18564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9282, 'num_write_kb': 37128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5159}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9195", 'reported_seq': 9273, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.120065+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:05.120065+0000', 'last_peered': '2024-09-11T01:18:05.120065+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:05.120065+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9195, 'log_dups_size': 0, 'ondisk_log_size': 9195, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37646336, 'num_objects': 9191, 'num_object_clones': 0, 'num_object_copies': 18382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7256, 'num_write_kb': 29024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5186}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9326", 'reported_seq': 9454, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.978556+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:05.978556+0000', 'last_peered': '2024-09-11T01:18:05.978556+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:05.978556+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9326, 'log_dups_size': 0, 'ondisk_log_size': 9326, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38179299, 'num_objects': 9322, 'num_object_clones': 0, 'num_object_copies': 18644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9322, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5814, 'num_write_kb': 23256, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5207}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:07.200 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9606", 'reported_seq': 9734, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.935803+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:05.935803+0000', 'last_peered': '2024-09-11T01:18:05.935803+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:05.935803+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9606, 'log_dups_size': 0, 'ondisk_log_size': 9606, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39322566, 'num_objects': 9602, 'num_object_clones': 0, 'num_object_copies': 19204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9602, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6359, 'num_write_kb': 25436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5319}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'9470", 'reported_seq': 9594, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.951638+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:05.951638+0000', 'last_peered': '2024-09-11T01:18:05.951638+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:05.951638+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 9470, 'log_dups_size': 0, 'ondisk_log_size': 9470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38756352, 'num_objects': 9462, 'num_object_clones': 0, 'num_object_copies': 18924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5834, 'num_write_kb': 23336, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5357}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9221", 'reported_seq': 9310, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.168117+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:05.168117+0000', 'last_peered': '2024-09-11T01:18:05.168117+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:05.168117+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9221, 'log_dups_size': 0, 'ondisk_log_size': 9221, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37749219, 'num_objects': 9217, 'num_object_clones': 0, 'num_object_copies': 18434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9217, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6990, 'num_write_kb': 27960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5190}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'9473", 'reported_seq': 9561, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.151550+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:05.151550+0000', 'last_peered': '2024-09-11T01:18:05.151550+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:05.151550+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 9473, 'log_dups_size': 0, 'ondisk_log_size': 9473, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38776832, 'num_objects': 9467, 'num_object_clones': 0, 'num_object_copies': 18934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7110, 'num_write_kb': 28440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5290}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'9408", 'reported_seq': 10410, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:06.015645+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:06.015645+0000', 'last_peered': '2024-09-11T01:18:06.015645+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:06.015645+0000', 'last_undegraded': '2024-09-11T01:18:06.015645+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 9408, 'log_dups_size': 0, 'ondisk_log_size': 9408, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38526976, 'num_objects': 9406, 'num_object_clones': 0, 'num_object_copies': 18812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3726, 'num_objects_unfound': 0, 'num_objects_dirty': 9406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6098, 'num_write_kb': 24392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 438, 'num_bytes_recovered': 1794048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5680}, {'shards': '0,2', 'objects': 3726}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:07.201 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9552", 'reported_seq': 9681, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.981044+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:05.981044+0000', 'last_peered': '2024-09-11T01:18:05.981044+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:05.981044+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9552, 'log_dups_size': 0, 'ondisk_log_size': 9552, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39092224, 'num_objects': 9544, 'num_object_clones': 0, 'num_object_copies': 19088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5945, 'num_write_kb': 23780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5352}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:07.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'9478", 'reported_seq': 9606, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.885609+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:05.885609+0000', 'last_peered': '2024-09-11T01:18:05.885609+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:05.885609+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 9478, 'log_dups_size': 0, 'ondisk_log_size': 9478, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38805504, 'num_objects': 9474, 'num_object_clones': 0, 'num_object_copies': 18948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9474, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6013, 'num_write_kb': 24052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5326}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.202 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:07.202 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9285", 'reported_seq': 9415, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:05.933756+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:05.933756+0000', 'last_peered': '2024-09-11T01:18:05.933756+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:05.933756+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9285, 'log_dups_size': 0, 'ondisk_log_size': 9285, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37983174, 'num_objects': 9275, 'num_object_clones': 0, 'num_object_copies': 18550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9275, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5942, 'num_write_kb': 23768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5121}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:07.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:07.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 211 20 150700 150680 2.78895 1.61328 0.0374355 0.0280051 2024-09-11T01:18:07.918 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:07.918 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/299920 objects degraded (13.785%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":288,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149960,"data_bytes":614191857,"bytes_used":2015633408,"bytes_avail":287894659072,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":299920,"degraded_ratio":0.13784675913576955,"misplaced_objects":3726,"misplaced_total":299920,"misplaced_ratio":0.012423312883435582,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":73026,"recovering_keys_per_sec":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"write_bytes_sec":2243002,"read_op_per_sec":0,"write_op_per_sec":547},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:07.943 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:07.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:08.356 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 212 20 151106 151086 2.78328 1.58594 0.0222899 0.0280608 2024-09-11T01:18:08.517 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:08.517 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:08.533 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-11T01:18:08.482731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614191857,"num_objects":149960,"num_object_clones":0,"num_object_copies":299920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149960,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109095,"num_write_kb":436374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18686454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150033,"ondisk_log_size":150033,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1968392,"kb_used_data":1213912,"kb_used_omap":12,"kb_used_meta":754419,"kb_avail":281147128,"statfs":{"total":289910292480,"available":287894659072,"internally_reserved":0,"allocated":1243045888,"data_stored":1242192463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":772525063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,14,8,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-146,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4443,"num_write_kb":17772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":598016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"2.f","version":"41'9282","reported_seq":9350,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.914268+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:05.914268+0000","last_peered":"2024-09-11T01:18:05.914268+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:05.914268+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9282,"num_write_kb":37128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9409","reported_seq":9483,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.055815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:05.055815+0000","last_peered":"2024-09-11T01:18:05.055815+0000","last_clean":"2024-09-11T01:18:05.055815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:05.055815+0000","last_undegraded":"2024-09-11T01:18:05.055815+0000","last_fullsized":"2024-09-11T01:18:05.055815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9409,"log_dups_size":0,"ondisk_log_size":9409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38515654,"num_objects":9405,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9405,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7507,"num_write_kb":30028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9195","reported_seq":9273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.120065+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:05.120065+0000","last_peered":"2024-09-11T01:18:05.120065+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:05.120065+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7256,"num_write_kb":29024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9359","reported_seq":9399,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.109015+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:05.109015+0000","last_peered":"2024-09-11T01:18:05.109015+0000","last_clean":"2024-09-11T01:18:05.109015+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:05.109015+0000","last_undegraded":"2024-09-11T01:18:05.109015+0000","last_fullsized":"2024-09-11T01:18:05.109015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9359,"log_dups_size":0,"ondisk_log_size":9359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38334464,"num_objects":9359,"num_object_clones":0,"num_object_copies":18718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9359,"num_write_kb":37436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9320","reported_seq":9434,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.981913+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:05.981913+0000","last_peered":"2024-09-11T01:18:05.981913+0000","last_clean":"2024-09-11T01:18:05.981913+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:05.981913+0000","last_undegraded":"2024-09-11T01:18:05.981913+0000","last_fullsized":"2024-09-11T01:18:05.981913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146531,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6118,"num_write_kb":24472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9326","reported_seq":9454,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.978556+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:05.978556+0000","last_peered":"2024-09-11T01:18:05.978556+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:05.978556+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9326,"log_dups_size":0,"ondisk_log_size":9326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9606","reported_seq":9734,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.935803+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:05.935803+0000","last_peered":"2024-09-11T01:18:05.935803+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:05.935803+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322566,"num_objects":9602,"num_object_clones":0,"num_object_copies":19204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9602,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5319},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9389","reported_seq":9503,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.988909+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:05.988909+0000","last_peered":"2024-09-11T01:18:05.988909+0000","last_clean":"2024-09-11T01:18:05.988909+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:05.988909+0000","last_undegraded":"2024-09-11T01:18:05.988909+0000","last_fullsized":"2024-09-11T01:18:05.988909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38449152,"num_objects":9387,"num_object_clones":0,"num_object_copies":18774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9257","reported_seq":17595,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:03.622835+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:03.622835+0000","last_peered":"2024-09-11T01:18:03.622835+0000","last_clean":"2024-09-11T01:18:03.622835+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:03.622835+0000","last_undegraded":"2024-09-11T01:18:03.622835+0000","last_fullsized":"2024-09-11T01:18:03.622835+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9257,"log_dups_size":0,"ondisk_log_size":9257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888483,"num_objects":9251,"num_object_clones":0,"num_object_copies":18502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7124,"num_write_kb":28496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9470","reported_seq":9594,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.951638+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:05.951638+0000","last_peered":"2024-09-11T01:18:05.951638+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:05.951638+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756352,"num_objects":9462,"num_object_clones":0,"num_object_copies":18924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5834,"num_write_kb":23336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5357},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9221","reported_seq":9310,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.168117+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:05.168117+0000","last_peered":"2024-09-11T01:18:05.168117+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:05.168117+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9221,"log_dups_size":0,"ondisk_log_size":9221,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37749219,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6990,"num_write_kb":27960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9473","reported_seq":9561,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.151550+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:05.151550+0000","last_peered":"2024-09-11T01:18:05.151550+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:05.151550+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9473,"log_dups_size":0,"ondisk_log_size":9473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38776832,"num_objects":9467,"num_object_clones":0,"num_object_copies":18934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7110,"num_write_kb":28440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5290}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9408","reported_seq":10410,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:06.015645+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:06.015645+0000","last_peered":"2024-09-11T01:18:06.015645+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:06.015645+0000","last_undegraded":"2024-09-11T01:18:06.015645+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9408,"log_dups_size":0,"ondisk_log_size":9408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6098,"num_write_kb":24392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":438,"num_bytes_recovered":1794048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5680},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9552","reported_seq":9681,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.981044+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:05.981044+0000","last_peered":"2024-09-11T01:18:05.981044+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:05.981044+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5945,"num_write_kb":23780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9478","reported_seq":9606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.885609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:05.885609+0000","last_peered":"2024-09-11T01:18:05.885609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:05.885609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9478,"log_dups_size":0,"ondisk_log_size":9478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9285","reported_seq":9415,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.933756+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:05.933756+0000","last_peered":"2024-09-11T01:18:05.933756+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:05.933756+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37983174,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5121},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":614191838,"num_objects":149958,"num_object_clones":0,"num_object_copies":299916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149958,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109093,"num_write_kb":436372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4564,"num_bytes_recovered":18686435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1244471296,"data_stored":1244319550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150030,"ondisk_log_size":150030,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679852,"kb_used_data":431580,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93691988,"statfs":{"total":96636764160,"available":95940595712,"internally_reserved":0,"allocated":441937920,"data_stored":441670615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789986,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474276,"kb_used_data":266260,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93897564,"statfs":{"total":96636764160,"available":96151105536,"internally_reserved":0,"allocated":272650240,"data_stored":272335502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814264,"kb_used_data":516072,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93557576,"statfs":{"total":96636764160,"available":95802957824,"internally_reserved":0,"allocated":528457728,"data_stored":528186346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,7,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529305600,"data_stored":529269470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273342464,"data_stored":273262978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441823232,"data_stored":441787102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:09.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 213 20 151519 151499 2.77778 1.61328 0.0504019 0.0281179 2024-09-11T01:18:10.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 214 20 151981 151961 2.77324 1.80469 0.0347147 0.028164 2024-09-11T01:18:11.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 215 20 152434 152414 2.76857 1.76953 0.121054 0.0282101 2024-09-11T01:18:11.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:12.111 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:12.112 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:12.135 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-11T01:18:10.483077+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617509617,"num_objects":150770,"num_object_clones":0,"num_object_copies":301540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":150770,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109905,"num_write_kb":439614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4565,"num_bytes_recovered":18686454,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150843,"ondisk_log_size":150843,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006032,"kb_used_data":1225184,"kb_used_omap":12,"kb_used_meta":780787,"kb_avail":281109488,"statfs":{"total":289910292480,"available":287856115712,"internally_reserved":0,"allocated":1254588416,"data_stored":1253734991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":799525895},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,11,7,7,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-146,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5253,"num_write_kb":21012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":146,"num_bytes_recovered":598016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002862"},"pg_stats":[{"pgid":"2.f","version":"41'9282","reported_seq":9350,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.914268+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:05.914268+0000","last_peered":"2024-09-11T01:18:05.914268+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:05.914268+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9282,"log_dups_size":0,"ondisk_log_size":9282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9282,"num_write_kb":37128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5159},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9553","reported_seq":9628,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.159630+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:10.159630+0000","last_peered":"2024-09-11T01:18:10.159630+0000","last_clean":"2024-09-11T01:18:10.159630+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:10.159630+0000","last_undegraded":"2024-09-11T01:18:10.159630+0000","last_fullsized":"2024-09-11T01:18:10.159630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39105478,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7651,"num_write_kb":30604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9335","reported_seq":9413,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.110161+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:10.110161+0000","last_peered":"2024-09-11T01:18:10.110161+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:10.110161+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38219776,"num_objects":9331,"num_object_clones":0,"num_object_copies":18662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7396,"num_write_kb":29584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5326}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9490","reported_seq":9530,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.117685+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:10.117685+0000","last_peered":"2024-09-11T01:18:10.117685+0000","last_clean":"2024-09-11T01:18:10.117685+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:10.117685+0000","last_undegraded":"2024-09-11T01:18:10.117685+0000","last_fullsized":"2024-09-11T01:18:10.117685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38871040,"num_objects":9490,"num_object_clones":0,"num_object_copies":18980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":37960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9320","reported_seq":9434,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.981913+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:05.981913+0000","last_peered":"2024-09-11T01:18:05.981913+0000","last_clean":"2024-09-11T01:18:05.981913+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:05.981913+0000","last_undegraded":"2024-09-11T01:18:05.981913+0000","last_fullsized":"2024-09-11T01:18:05.981913+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9320,"log_dups_size":0,"ondisk_log_size":9320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146531,"num_objects":9314,"num_object_clones":0,"num_object_copies":18628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6118,"num_write_kb":24472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9326","reported_seq":9454,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.978556+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:05.978556+0000","last_peered":"2024-09-11T01:18:05.978556+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:05.978556+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9326,"log_dups_size":0,"ondisk_log_size":9326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5814,"num_write_kb":23256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5207},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9606","reported_seq":9734,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.935803+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:05.935803+0000","last_peered":"2024-09-11T01:18:05.935803+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:05.935803+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9606,"log_dups_size":0,"ondisk_log_size":9606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39322566,"num_objects":9602,"num_object_clones":0,"num_object_copies":19204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9602,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5319},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9389","reported_seq":9503,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.988909+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:05.988909+0000","last_peered":"2024-09-11T01:18:05.988909+0000","last_clean":"2024-09-11T01:18:05.988909+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:05.988909+0000","last_undegraded":"2024-09-11T01:18:05.988909+0000","last_fullsized":"2024-09-11T01:18:05.988909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9389,"log_dups_size":0,"ondisk_log_size":9389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38449152,"num_objects":9387,"num_object_clones":0,"num_object_copies":18774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6342,"num_write_kb":25368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9382","reported_seq":17720,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:08.697671+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:08.697671+0000","last_peered":"2024-09-11T01:18:08.697671+0000","last_clean":"2024-09-11T01:18:08.697671+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:08.697671+0000","last_undegraded":"2024-09-11T01:18:08.697671+0000","last_fullsized":"2024-09-11T01:18:08.697671+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9382,"log_dups_size":0,"ondisk_log_size":9382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38400483,"num_objects":9376,"num_object_clones":0,"num_object_copies":18752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9470","reported_seq":9594,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.951638+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:05.951638+0000","last_peered":"2024-09-11T01:18:05.951638+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:05.951638+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9470,"log_dups_size":0,"ondisk_log_size":9470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756352,"num_objects":9462,"num_object_clones":0,"num_object_copies":18924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5834,"num_write_kb":23336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5357},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9352","reported_seq":9441,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.065683+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:10.065683+0000","last_peered":"2024-09-11T01:18:10.065683+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:10.065683+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9352,"log_dups_size":0,"ondisk_log_size":9352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285795,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7121,"num_write_kb":28484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9612","reported_seq":9700,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.098205+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:10.098205+0000","last_peered":"2024-09-11T01:18:10.098205+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:10.098205+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9612,"log_dups_size":0,"ondisk_log_size":9612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9408","reported_seq":10410,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:06.015645+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:06.015645+0000","last_peered":"2024-09-11T01:18:06.015645+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:06.015645+0000","last_undegraded":"2024-09-11T01:18:06.015645+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9408,"log_dups_size":0,"ondisk_log_size":9408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6098,"num_write_kb":24392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":438,"num_bytes_recovered":1794048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5680},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9552","reported_seq":9681,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.981044+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:05.981044+0000","last_peered":"2024-09-11T01:18:05.981044+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:05.981044+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9552,"log_dups_size":0,"ondisk_log_size":9552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5945,"num_write_kb":23780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5352},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9478","reported_seq":9606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.885609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:05.885609+0000","last_peered":"2024-09-11T01:18:05.885609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:05.885609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9478,"log_dups_size":0,"ondisk_log_size":9478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38805504,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6013,"num_write_kb":24052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9285","reported_seq":9415,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:05.933756+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:05.933756+0000","last_peered":"2024-09-11T01:18:05.933756+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:05.933756+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9285,"log_dups_size":0,"ondisk_log_size":9285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37983174,"num_objects":9275,"num_object_clones":0,"num_object_copies":18550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5942,"num_write_kb":23768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5121},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617509598,"num_objects":150768,"num_object_clones":0,"num_object_copies":301536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":150768,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":109903,"num_write_kb":439612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4564,"num_bytes_recovered":18686435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1255927808,"data_stored":1255776062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150840,"ondisk_log_size":150840,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684180,"kb_used_data":435908,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93687660,"statfs":{"total":96636764160,"available":95936163840,"internally_reserved":0,"allocated":446369792,"data_stored":446102487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507588,"kb_used_data":273204,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93864252,"statfs":{"total":96636764160,"available":96116994048,"internally_reserved":0,"allocated":279760896,"data_stored":279446158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":814264,"kb_used_data":516072,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93557576,"statfs":{"total":96636764160,"available":95802957824,"internally_reserved":0,"allocated":528457728,"data_stored":528186346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,7,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529305600,"data_stored":529269470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":280346624,"data_stored":280267138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446275584,"data_stored":446239454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:12.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:12.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 216 20 152854 152834 2.76335 1.64062 0.0500388 0.0282646 2024-09-11T01:18:12.729 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:12.730 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:12.764 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-11T01:18:12.483572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623026929,"num_objects":152117,"num_object_clones":0,"num_object_copies":304234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3652,"num_objects_unfound":0,"num_objects_dirty":152117,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":111252,"num_write_kb":445002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4639,"num_bytes_recovered":18989558,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152190,"ondisk_log_size":152190,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016556,"kb_used_data":1231548,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281098964,"statfs":{"total":289910292480,"available":287845339136,"internally_reserved":0,"allocated":1261105152,"data_stored":1260251727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":803785735},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,6,5,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23584768,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5758,"num_write_kb":23032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"2.f","version":"41'9422","reported_seq":9490,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.017595+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:11.017595+0000","last_peered":"2024-09-11T01:18:11.017595+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:11.017595+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9422,"log_dups_size":0,"ondisk_log_size":9422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38592512,"num_objects":9422,"num_object_clones":0,"num_object_copies":18844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9422,"num_write_kb":37688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5299},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9553","reported_seq":9628,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.159630+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:10.159630+0000","last_peered":"2024-09-11T01:18:10.159630+0000","last_clean":"2024-09-11T01:18:10.159630+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:10.159630+0000","last_undegraded":"2024-09-11T01:18:10.159630+0000","last_fullsized":"2024-09-11T01:18:10.159630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39105478,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7651,"num_write_kb":30604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9335","reported_seq":9413,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.110161+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:10.110161+0000","last_peered":"2024-09-11T01:18:10.110161+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:10.110161+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38219776,"num_objects":9331,"num_object_clones":0,"num_object_copies":18662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7396,"num_write_kb":29584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5326}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9490","reported_seq":9530,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.117685+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:10.117685+0000","last_peered":"2024-09-11T01:18:10.117685+0000","last_clean":"2024-09-11T01:18:10.117685+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:10.117685+0000","last_undegraded":"2024-09-11T01:18:10.117685+0000","last_fullsized":"2024-09-11T01:18:10.117685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38871040,"num_objects":9490,"num_object_clones":0,"num_object_copies":18980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":37960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9449","reported_seq":9565,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.051530+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:11.051530+0000","last_peered":"2024-09-11T01:18:11.051530+0000","last_clean":"2024-09-11T01:18:11.051530+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:11.051530+0000","last_undegraded":"2024-09-11T01:18:11.051530+0000","last_fullsized":"2024-09-11T01:18:11.051530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9449,"log_dups_size":0,"ondisk_log_size":9449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674915,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9456","reported_seq":9585,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.005964+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:11.005964+0000","last_peered":"2024-09-11T01:18:11.005964+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:11.005964+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9456,"log_dups_size":0,"ondisk_log_size":9456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38711779,"num_objects":9452,"num_object_clones":0,"num_object_copies":18904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5337},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9730","reported_seq":9858,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.958861+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:10.958861+0000","last_peered":"2024-09-11T01:18:10.958861+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:10.958861+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6483,"num_write_kb":25932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5443},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9520","reported_seq":9633,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.051175+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:11.051175+0000","last_peered":"2024-09-11T01:18:11.051175+0000","last_clean":"2024-09-11T01:18:11.051175+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:11.051175+0000","last_undegraded":"2024-09-11T01:18:11.051175+0000","last_fullsized":"2024-09-11T01:18:11.051175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38985728,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6473,"num_write_kb":25892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9382","reported_seq":17720,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:08.697671+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:08.697671+0000","last_peered":"2024-09-11T01:18:08.697671+0000","last_clean":"2024-09-11T01:18:08.697671+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:08.697671+0000","last_undegraded":"2024-09-11T01:18:08.697671+0000","last_fullsized":"2024-09-11T01:18:08.697671+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9382,"log_dups_size":0,"ondisk_log_size":9382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38400483,"num_objects":9376,"num_object_clones":0,"num_object_copies":18752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9605","reported_seq":9729,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.015078+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:11.015078+0000","last_peered":"2024-09-11T01:18:11.015078+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:11.015078+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9605,"log_dups_size":0,"ondisk_log_size":9605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39309312,"num_objects":9597,"num_object_clones":0,"num_object_copies":19194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9352","reported_seq":9441,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.065683+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:10.065683+0000","last_peered":"2024-09-11T01:18:10.065683+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:10.065683+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9352,"log_dups_size":0,"ondisk_log_size":9352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285795,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7121,"num_write_kb":28484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9612","reported_seq":9700,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.098205+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:10.098205+0000","last_peered":"2024-09-11T01:18:10.098205+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:10.098205+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9612,"log_dups_size":0,"ondisk_log_size":9612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9546","reported_seq":10692,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:11.056322+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:11.056322+0000","last_peered":"2024-09-11T01:18:11.056322+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:11.056322+0000","last_undegraded":"2024-09-11T01:18:11.056322+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3652,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6236,"num_write_kb":24944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5892},{"shards":"0,2","objects":3652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9689","reported_seq":9818,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.004889+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:11.004889+0000","last_peered":"2024-09-11T01:18:11.004889+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:11.004889+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9689,"log_dups_size":0,"ondisk_log_size":9689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39653376,"num_objects":9681,"num_object_clones":0,"num_object_copies":19362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6082,"num_write_kb":24328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9620","reported_seq":9748,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.074069+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:11.074069+0000","last_peered":"2024-09-11T01:18:11.074069+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:11.074069+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39387136,"num_objects":9616,"num_object_clones":0,"num_object_copies":19232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5468},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9426","reported_seq":9556,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.043148+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:11.043148+0000","last_peered":"2024-09-11T01:18:11.043148+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:11.043148+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38560710,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6083,"num_write_kb":24332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623026910,"num_objects":152115,"num_object_clones":0,"num_object_copies":304230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3652,"num_objects_unfound":0,"num_objects_dirty":152115,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":111250,"num_write_kb":445000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4638,"num_bytes_recovered":18989539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262555136,"data_stored":1262403390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152187,"ondisk_log_size":152187,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684180,"kb_used_data":435908,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93687660,"statfs":{"total":96636764160,"available":95936163840,"internally_reserved":0,"allocated":446369792,"data_stored":446102487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507588,"kb_used_data":273204,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93864252,"statfs":{"total":96636764160,"available":96116994048,"internally_reserved":0,"allocated":279760896,"data_stored":279446158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824788,"kb_used_data":522436,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93547052,"statfs":{"total":96636764160,"available":95792181248,"internally_reserved":0,"allocated":534974464,"data_stored":534703082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535932928,"data_stored":535896798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":280346624,"data_stored":280267138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446275584,"data_stored":446239454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'9422", 'reported_seq': 9490, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:11.017595+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:11.017595+0000', 'last_peered': '2024-09-11T01:18:11.017595+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:11.017595+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 9422, 'log_dups_size': 0, 'ondisk_log_size': 9422, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38592512, 'num_objects': 9422, 'num_object_clones': 0, 'num_object_copies': 18844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9422, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9422, 'num_write_kb': 37688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5299}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9335", 'reported_seq': 9413, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:10.110161+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:10.110161+0000', 'last_peered': '2024-09-11T01:18:10.110161+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:10.110161+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9335, 'log_dups_size': 0, 'ondisk_log_size': 9335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38219776, 'num_objects': 9331, 'num_object_clones': 0, 'num_object_copies': 18662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7396, 'num_write_kb': 29584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5326}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9456", 'reported_seq': 9585, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:11.005964+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:11.005964+0000', 'last_peered': '2024-09-11T01:18:11.005964+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:11.005964+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9456, 'log_dups_size': 0, 'ondisk_log_size': 9456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38711779, 'num_objects': 9452, 'num_object_clones': 0, 'num_object_copies': 18904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9452, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5944, 'num_write_kb': 23776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5337}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.766 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9730", 'reported_seq': 9858, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:10.958861+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:10.958861+0000', 'last_peered': '2024-09-11T01:18:10.958861+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:10.958861+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9730, 'log_dups_size': 0, 'ondisk_log_size': 9730, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39830470, 'num_objects': 9726, 'num_object_clones': 0, 'num_object_copies': 19452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9726, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6483, 'num_write_kb': 25932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5443}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'9605", 'reported_seq': 9729, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:11.015078+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:11.015078+0000', 'last_peered': '2024-09-11T01:18:11.015078+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:11.015078+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 9605, 'log_dups_size': 0, 'ondisk_log_size': 9605, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39309312, 'num_objects': 9597, 'num_object_clones': 0, 'num_object_copies': 19194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5969, 'num_write_kb': 23876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5492}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9352", 'reported_seq': 9441, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:10.065683+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:10.065683+0000', 'last_peered': '2024-09-11T01:18:10.065683+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:10.065683+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9352, 'log_dups_size': 0, 'ondisk_log_size': 9352, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38285795, 'num_objects': 9348, 'num_object_clones': 0, 'num_object_copies': 18696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9348, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7121, 'num_write_kb': 28484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5321}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'9612", 'reported_seq': 9700, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:10.098205+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:10.098205+0000', 'last_peered': '2024-09-11T01:18:10.098205+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:10.098205+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 9612, 'log_dups_size': 0, 'ondisk_log_size': 9612, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39346176, 'num_objects': 9606, 'num_object_clones': 0, 'num_object_copies': 19212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7249, 'num_write_kb': 28996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5429}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'9546", 'reported_seq': 10692, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:11.056322+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:11.056322+0000', 'last_peered': '2024-09-11T01:18:11.056322+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:11.056322+0000', 'last_undegraded': '2024-09-11T01:18:11.056322+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 9546, 'log_dups_size': 0, 'ondisk_log_size': 9546, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39092224, 'num_objects': 9544, 'num_object_clones': 0, 'num_object_copies': 19088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3652, 'num_objects_unfound': 0, 'num_objects_dirty': 9544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6236, 'num_write_kb': 24944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 512, 'num_bytes_recovered': 2097152, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5892}, {'shards': '0,2', 'objects': 3652}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:12.767 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9689", 'reported_seq': 9818, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:11.004889+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:11.004889+0000', 'last_peered': '2024-09-11T01:18:11.004889+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:11.004889+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9689, 'log_dups_size': 0, 'ondisk_log_size': 9689, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39653376, 'num_objects': 9681, 'num_object_clones': 0, 'num_object_copies': 19362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6082, 'num_write_kb': 24328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5489}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:12.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'9620", 'reported_seq': 9748, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:11.074069+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:11.074069+0000', 'last_peered': '2024-09-11T01:18:11.074069+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:11.074069+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 9620, 'log_dups_size': 0, 'ondisk_log_size': 9620, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39387136, 'num_objects': 9616, 'num_object_clones': 0, 'num_object_copies': 19232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6155, 'num_write_kb': 24620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5468}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.768 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:12.768 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9426", 'reported_seq': 9556, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:11.043148+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:11.043148+0000', 'last_peered': '2024-09-11T01:18:11.043148+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:11.043148+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9426, 'log_dups_size': 0, 'ondisk_log_size': 9426, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38560710, 'num_objects': 9416, 'num_object_clones': 0, 'num_object_copies': 18832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9416, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6083, 'num_write_kb': 24332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5262}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:12.768 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:13.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 217 20 153264 153244 2.75799 1.60156 0.12966 0.0283172 2024-09-11T01:18:13.493 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:13.493 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/304234 objects degraded (13.589%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":293,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152117,"data_bytes":623026929,"bytes_used":2064953344,"bytes_avail":287845339136,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":304234,"degraded_ratio":0.13589210936318755,"misplaced_objects":3652,"misplaced_total":304234,"misplaced_ratio":0.012003918036774325,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75075,"recovering_keys_per_sec":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"write_bytes_sec":1964919,"read_op_per_sec":0,"write_op_per_sec":479},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:13.517 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:13.517 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:14.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:14.096 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:14.124 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-11T01:18:12.483572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623026929,"num_objects":152117,"num_object_clones":0,"num_object_copies":304234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3652,"num_objects_unfound":0,"num_objects_dirty":152117,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":111252,"num_write_kb":445002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4639,"num_bytes_recovered":18989558,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152190,"ondisk_log_size":152190,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016556,"kb_used_data":1231548,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281098964,"statfs":{"total":289910292480,"available":287845339136,"internally_reserved":0,"allocated":1261105152,"data_stored":1260251727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":803785735},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,6,5,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23584768,"num_objects":5758,"num_object_clones":0,"num_object_copies":11516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":5758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5758,"num_write_kb":23032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002917"},"pg_stats":[{"pgid":"2.f","version":"41'9422","reported_seq":9490,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.017595+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:11.017595+0000","last_peered":"2024-09-11T01:18:11.017595+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:11.017595+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9422,"log_dups_size":0,"ondisk_log_size":9422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38592512,"num_objects":9422,"num_object_clones":0,"num_object_copies":18844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9422,"num_write_kb":37688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5299},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9553","reported_seq":9628,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.159630+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:10.159630+0000","last_peered":"2024-09-11T01:18:10.159630+0000","last_clean":"2024-09-11T01:18:10.159630+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:10.159630+0000","last_undegraded":"2024-09-11T01:18:10.159630+0000","last_fullsized":"2024-09-11T01:18:10.159630+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39105478,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7651,"num_write_kb":30604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9335","reported_seq":9413,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.110161+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:10.110161+0000","last_peered":"2024-09-11T01:18:10.110161+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:10.110161+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9335,"log_dups_size":0,"ondisk_log_size":9335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38219776,"num_objects":9331,"num_object_clones":0,"num_object_copies":18662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7396,"num_write_kb":29584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5326}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9490","reported_seq":9530,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.117685+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:10.117685+0000","last_peered":"2024-09-11T01:18:10.117685+0000","last_clean":"2024-09-11T01:18:10.117685+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:10.117685+0000","last_undegraded":"2024-09-11T01:18:10.117685+0000","last_fullsized":"2024-09-11T01:18:10.117685+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9490,"log_dups_size":0,"ondisk_log_size":9490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38871040,"num_objects":9490,"num_object_clones":0,"num_object_copies":18980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9490,"num_write_kb":37960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9449","reported_seq":9565,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.051530+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:11.051530+0000","last_peered":"2024-09-11T01:18:11.051530+0000","last_clean":"2024-09-11T01:18:11.051530+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:11.051530+0000","last_undegraded":"2024-09-11T01:18:11.051530+0000","last_fullsized":"2024-09-11T01:18:11.051530+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9449,"log_dups_size":0,"ondisk_log_size":9449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674915,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9456","reported_seq":9585,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.005964+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:11.005964+0000","last_peered":"2024-09-11T01:18:11.005964+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:11.005964+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9456,"log_dups_size":0,"ondisk_log_size":9456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38711779,"num_objects":9452,"num_object_clones":0,"num_object_copies":18904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5944,"num_write_kb":23776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5337},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9730","reported_seq":9858,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.958861+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:10.958861+0000","last_peered":"2024-09-11T01:18:10.958861+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:10.958861+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9730,"log_dups_size":0,"ondisk_log_size":9730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6483,"num_write_kb":25932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5443},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9520","reported_seq":9633,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.051175+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:11.051175+0000","last_peered":"2024-09-11T01:18:11.051175+0000","last_clean":"2024-09-11T01:18:11.051175+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:11.051175+0000","last_undegraded":"2024-09-11T01:18:11.051175+0000","last_fullsized":"2024-09-11T01:18:11.051175+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38985728,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6473,"num_write_kb":25892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9382","reported_seq":17720,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:08.697671+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:08.697671+0000","last_peered":"2024-09-11T01:18:08.697671+0000","last_clean":"2024-09-11T01:18:08.697671+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:08.697671+0000","last_undegraded":"2024-09-11T01:18:08.697671+0000","last_fullsized":"2024-09-11T01:18:08.697671+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9382,"log_dups_size":0,"ondisk_log_size":9382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38400483,"num_objects":9376,"num_object_clones":0,"num_object_copies":18752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9376,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9605","reported_seq":9729,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.015078+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:11.015078+0000","last_peered":"2024-09-11T01:18:11.015078+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:11.015078+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9605,"log_dups_size":0,"ondisk_log_size":9605,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39309312,"num_objects":9597,"num_object_clones":0,"num_object_copies":19194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9352","reported_seq":9441,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.065683+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:10.065683+0000","last_peered":"2024-09-11T01:18:10.065683+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:10.065683+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9352,"log_dups_size":0,"ondisk_log_size":9352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285795,"num_objects":9348,"num_object_clones":0,"num_object_copies":18696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7121,"num_write_kb":28484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9612","reported_seq":9700,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:10.098205+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:10.098205+0000","last_peered":"2024-09-11T01:18:10.098205+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:10.098205+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9612,"log_dups_size":0,"ondisk_log_size":9612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5429}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9546","reported_seq":10692,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:11.056322+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:11.056322+0000","last_peered":"2024-09-11T01:18:11.056322+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:11.056322+0000","last_undegraded":"2024-09-11T01:18:11.056322+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9546,"log_dups_size":0,"ondisk_log_size":9546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3652,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6236,"num_write_kb":24944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":512,"num_bytes_recovered":2097152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5892},{"shards":"0,2","objects":3652}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9689","reported_seq":9818,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.004889+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:11.004889+0000","last_peered":"2024-09-11T01:18:11.004889+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:11.004889+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9689,"log_dups_size":0,"ondisk_log_size":9689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39653376,"num_objects":9681,"num_object_clones":0,"num_object_copies":19362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6082,"num_write_kb":24328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5489},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9620","reported_seq":9748,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.074069+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:11.074069+0000","last_peered":"2024-09-11T01:18:11.074069+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:11.074069+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39387136,"num_objects":9616,"num_object_clones":0,"num_object_copies":19232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5468},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9426","reported_seq":9556,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.043148+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:11.043148+0000","last_peered":"2024-09-11T01:18:11.043148+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:11.043148+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9426,"log_dups_size":0,"ondisk_log_size":9426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38560710,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6083,"num_write_kb":24332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5262},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623026910,"num_objects":152115,"num_object_clones":0,"num_object_copies":304230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3652,"num_objects_unfound":0,"num_objects_dirty":152115,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":111250,"num_write_kb":445000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4638,"num_bytes_recovered":18989539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1262555136,"data_stored":1262403390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152187,"ondisk_log_size":152187,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684180,"kb_used_data":435908,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93687660,"statfs":{"total":96636764160,"available":95936163840,"internally_reserved":0,"allocated":446369792,"data_stored":446102487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789987,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507588,"kb_used_data":273204,"kb_used_omap":4,"kb_used_meta":234363,"kb_avail":93864252,"statfs":{"total":96636764160,"available":96116994048,"internally_reserved":0,"allocated":279760896,"data_stored":279446158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239988399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824788,"kb_used_data":522436,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93547052,"statfs":{"total":96636764160,"available":95792181248,"internally_reserved":0,"allocated":534974464,"data_stored":534703082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,2,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":535932928,"data_stored":535896798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":280346624,"data_stored":280267138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446275584,"data_stored":446239454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:14.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 218 20 153688 153668 2.75294 1.65625 0.0382825 0.0283718 2024-09-11T01:18:15.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 219 20 154142 154122 2.74847 1.77344 0.0220369 0.0284158 2024-09-11T01:18:16.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:2024-09-11T01:18:16.356684+0000 min lat: 0.00221854 max lat: 2.62159 avg lat: 0.0284626 2024-09-11T01:18:16.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:18:16.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 220 20 154599 154579 2.74409 1.78516 0.0374418 0.0284626 2024-09-11T01:18:17.127 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:17.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 221 20 155053 155033 2.73969 1.77344 0.0488858 0.0285087 2024-09-11T01:18:17.712 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:17.713 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:17.734 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T01:18:16.486355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632050417,"num_objects":154320,"num_object_clones":0,"num_object_copies":308640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":154320,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":113455,"num_write_kb":453814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4701,"num_bytes_recovered":19243510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154393,"ondisk_log_size":154393,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065960,"kb_used_data":1250424,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281049560,"statfs":{"total":289910292480,"available":287794749440,"internally_reserved":0,"allocated":1280434176,"data_stored":1279580751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":835046407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,11,16,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26198016,"num_objects":6396,"num_object_clones":0,"num_object_copies":12792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6396,"num_write_kb":25584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004685"},"pg_stats":[{"pgid":"2.f","version":"41'9580","reported_seq":9647,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.159207+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:16.159207+0000","last_peered":"2024-09-11T01:18:16.159207+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:16.159207+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9580,"log_dups_size":0,"ondisk_log_size":9580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39239680,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9580,"num_write_kb":38320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5457},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9675","reported_seq":9747,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.195391+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:15.195391+0000","last_peered":"2024-09-11T01:18:15.195391+0000","last_clean":"2024-09-11T01:18:15.195391+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:15.195391+0000","last_undegraded":"2024-09-11T01:18:15.195391+0000","last_fullsized":"2024-09-11T01:18:15.195391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9675,"log_dups_size":0,"ondisk_log_size":9675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39605190,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7773,"num_write_kb":31092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9487","reported_seq":9565,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.183147+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:15.183147+0000","last_peered":"2024-09-11T01:18:15.183147+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:15.183147+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9487,"log_dups_size":0,"ondisk_log_size":9487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7548,"num_write_kb":30192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5478}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9622","reported_seq":9662,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.131474+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:15.131474+0000","last_peered":"2024-09-11T01:18:15.131474+0000","last_clean":"2024-09-11T01:18:15.131474+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:15.131474+0000","last_undegraded":"2024-09-11T01:18:15.131474+0000","last_fullsized":"2024-09-11T01:18:15.131474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9622,"log_dups_size":0,"ondisk_log_size":9622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39411712,"num_objects":9622,"num_object_clones":0,"num_object_copies":19244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9622,"num_write_kb":38488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9601","reported_seq":9717,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.119555+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:16.119555+0000","last_peered":"2024-09-11T01:18:16.119555+0000","last_clean":"2024-09-11T01:18:16.119555+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:16.119555+0000","last_undegraded":"2024-09-11T01:18:16.119555+0000","last_fullsized":"2024-09-11T01:18:16.119555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9601,"log_dups_size":0,"ondisk_log_size":9601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297507,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6399,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9585","reported_seq":9714,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.104225+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:16.104225+0000","last_peered":"2024-09-11T01:18:16.104225+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:16.104225+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9585,"log_dups_size":0,"ondisk_log_size":9585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6073,"num_write_kb":24292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9853","reported_seq":9980,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.053622+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:16.053622+0000","last_peered":"2024-09-11T01:18:16.053622+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:16.053622+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9853,"log_dups_size":0,"ondisk_log_size":9853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6606,"num_write_kb":26424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5566},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9660","reported_seq":9774,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.104023+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:16.104023+0000","last_peered":"2024-09-11T01:18:16.104023+0000","last_clean":"2024-09-11T01:18:16.104023+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:16.104023+0000","last_undegraded":"2024-09-11T01:18:16.104023+0000","last_fullsized":"2024-09-11T01:18:16.104023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9660,"log_dups_size":0,"ondisk_log_size":9660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559168,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9524","reported_seq":17862,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:13.737990+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:13.737990+0000","last_peered":"2024-09-11T01:18:13.737990+0000","last_clean":"2024-09-11T01:18:13.737990+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:13.737990+0000","last_undegraded":"2024-09-11T01:18:13.737990+0000","last_fullsized":"2024-09-11T01:18:13.737990+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9524,"log_dups_size":0,"ondisk_log_size":9524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7391,"num_write_kb":29564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9749","reported_seq":9873,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.084674+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:16.084674+0000","last_peered":"2024-09-11T01:18:16.084674+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:16.084674+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9480","reported_seq":9569,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.107312+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:15.107312+0000","last_peered":"2024-09-11T01:18:15.107312+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:15.107312+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9480,"log_dups_size":0,"ondisk_log_size":9480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810083,"num_objects":9476,"num_object_clones":0,"num_object_copies":18952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5449}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9750","reported_seq":9838,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.166172+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:15.166172+0000","last_peered":"2024-09-11T01:18:15.166172+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:15.166172+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9750,"log_dups_size":0,"ondisk_log_size":9750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":9744,"num_object_clones":0,"num_object_copies":19488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7387,"num_write_kb":29548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9677","reported_seq":10951,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:16.110711+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:16.110711+0000","last_peered":"2024-09-11T01:18:16.110711+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:16.110711+0000","last_undegraded":"2024-09-11T01:18:16.110711+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628800,"num_objects":9675,"num_object_clones":0,"num_object_copies":19350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":9675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6367,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6085},{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9815","reported_seq":9944,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.134147+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:16.134147+0000","last_peered":"2024-09-11T01:18:16.134147+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:16.134147+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9815,"log_dups_size":0,"ondisk_log_size":9815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169472,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6208,"num_write_kb":24832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9770","reported_seq":9898,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.046167+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:16.046167+0000","last_peered":"2024-09-11T01:18:16.046167+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:16.046167+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9770,"log_dups_size":0,"ondisk_log_size":9770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40001536,"num_objects":9766,"num_object_clones":0,"num_object_copies":19532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6305,"num_write_kb":25220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5618},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9562","reported_seq":9692,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.084422+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:16.084422+0000","last_peered":"2024-09-11T01:18:16.084422+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:16.084422+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9562,"log_dups_size":0,"ondisk_log_size":9562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39117766,"num_objects":9552,"num_object_clones":0,"num_object_copies":19104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6219,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5398},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632050398,"num_objects":154318,"num_object_clones":0,"num_object_copies":308636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":154318,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":113453,"num_write_kb":453812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4700,"num_bytes_recovered":19243491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280888832,"data_stored":1280737086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154390,"ondisk_log_size":154390,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692644,"kb_used_data":440212,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93679196,"statfs":{"total":96636764160,"available":95927496704,"internally_reserved":0,"allocated":450777088,"data_stored":450509783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518548,"kb_used_data":280004,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93853292,"statfs":{"total":96636764160,"available":96105771008,"internally_reserved":0,"allocated":286724096,"data_stored":286409358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":854768,"kb_used_data":530208,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93517072,"statfs":{"total":96636764160,"available":95761481728,"internally_reserved":0,"allocated":542932992,"data_stored":542661610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":542711808,"data_stored":542675678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":287469568,"data_stored":287390082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450707456,"data_stored":450671326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:17.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:18.357 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 222 20 155442 155422 2.7342 1.51953 0.0665024 0.0285656 2024-09-11T01:18:18.377 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:18.378 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:18.404 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T01:18:16.486355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632050417,"num_objects":154320,"num_object_clones":0,"num_object_copies":308640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":154320,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":113455,"num_write_kb":453814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4701,"num_bytes_recovered":19243510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154393,"ondisk_log_size":154393,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065960,"kb_used_data":1250424,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281049560,"statfs":{"total":289910292480,"available":287794749440,"internally_reserved":0,"allocated":1280434176,"data_stored":1279580751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":835046407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,11,16,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26198016,"num_objects":6396,"num_object_clones":0,"num_object_copies":12792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":6396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6396,"num_write_kb":25584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004685"},"pg_stats":[{"pgid":"2.f","version":"41'9580","reported_seq":9647,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.159207+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:16.159207+0000","last_peered":"2024-09-11T01:18:16.159207+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:16.159207+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9580,"log_dups_size":0,"ondisk_log_size":9580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39239680,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9580,"num_write_kb":38320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5457},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9675","reported_seq":9747,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.195391+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:15.195391+0000","last_peered":"2024-09-11T01:18:15.195391+0000","last_clean":"2024-09-11T01:18:15.195391+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:15.195391+0000","last_undegraded":"2024-09-11T01:18:15.195391+0000","last_fullsized":"2024-09-11T01:18:15.195391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9675,"log_dups_size":0,"ondisk_log_size":9675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39605190,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7773,"num_write_kb":31092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9487","reported_seq":9565,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.183147+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:15.183147+0000","last_peered":"2024-09-11T01:18:15.183147+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:15.183147+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9487,"log_dups_size":0,"ondisk_log_size":9487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7548,"num_write_kb":30192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5478}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9622","reported_seq":9662,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.131474+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:15.131474+0000","last_peered":"2024-09-11T01:18:15.131474+0000","last_clean":"2024-09-11T01:18:15.131474+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:15.131474+0000","last_undegraded":"2024-09-11T01:18:15.131474+0000","last_fullsized":"2024-09-11T01:18:15.131474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9622,"log_dups_size":0,"ondisk_log_size":9622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39411712,"num_objects":9622,"num_object_clones":0,"num_object_copies":19244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9622,"num_write_kb":38488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9601","reported_seq":9717,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.119555+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:16.119555+0000","last_peered":"2024-09-11T01:18:16.119555+0000","last_clean":"2024-09-11T01:18:16.119555+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:16.119555+0000","last_undegraded":"2024-09-11T01:18:16.119555+0000","last_fullsized":"2024-09-11T01:18:16.119555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9601,"log_dups_size":0,"ondisk_log_size":9601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297507,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6399,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9585","reported_seq":9714,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.104225+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:16.104225+0000","last_peered":"2024-09-11T01:18:16.104225+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:16.104225+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9585,"log_dups_size":0,"ondisk_log_size":9585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6073,"num_write_kb":24292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9853","reported_seq":9980,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.053622+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:16.053622+0000","last_peered":"2024-09-11T01:18:16.053622+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:16.053622+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9853,"log_dups_size":0,"ondisk_log_size":9853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6606,"num_write_kb":26424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5566},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9660","reported_seq":9774,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.104023+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:16.104023+0000","last_peered":"2024-09-11T01:18:16.104023+0000","last_clean":"2024-09-11T01:18:16.104023+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:16.104023+0000","last_undegraded":"2024-09-11T01:18:16.104023+0000","last_fullsized":"2024-09-11T01:18:16.104023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9660,"log_dups_size":0,"ondisk_log_size":9660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559168,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9524","reported_seq":17862,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:13.737990+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:13.737990+0000","last_peered":"2024-09-11T01:18:13.737990+0000","last_clean":"2024-09-11T01:18:13.737990+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:13.737990+0000","last_undegraded":"2024-09-11T01:18:13.737990+0000","last_fullsized":"2024-09-11T01:18:13.737990+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9524,"log_dups_size":0,"ondisk_log_size":9524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7391,"num_write_kb":29564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9749","reported_seq":9873,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.084674+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:16.084674+0000","last_peered":"2024-09-11T01:18:16.084674+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:16.084674+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9480","reported_seq":9569,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.107312+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:15.107312+0000","last_peered":"2024-09-11T01:18:15.107312+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:15.107312+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9480,"log_dups_size":0,"ondisk_log_size":9480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810083,"num_objects":9476,"num_object_clones":0,"num_object_copies":18952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5449}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9750","reported_seq":9838,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.166172+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:15.166172+0000","last_peered":"2024-09-11T01:18:15.166172+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:15.166172+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9750,"log_dups_size":0,"ondisk_log_size":9750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":9744,"num_object_clones":0,"num_object_copies":19488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7387,"num_write_kb":29548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9677","reported_seq":10951,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:16.110711+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:16.110711+0000","last_peered":"2024-09-11T01:18:16.110711+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:16.110711+0000","last_undegraded":"2024-09-11T01:18:16.110711+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628800,"num_objects":9675,"num_object_clones":0,"num_object_copies":19350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":9675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6367,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6085},{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9815","reported_seq":9944,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.134147+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:16.134147+0000","last_peered":"2024-09-11T01:18:16.134147+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:16.134147+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9815,"log_dups_size":0,"ondisk_log_size":9815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169472,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6208,"num_write_kb":24832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9770","reported_seq":9898,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.046167+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:16.046167+0000","last_peered":"2024-09-11T01:18:16.046167+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:16.046167+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9770,"log_dups_size":0,"ondisk_log_size":9770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40001536,"num_objects":9766,"num_object_clones":0,"num_object_copies":19532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6305,"num_write_kb":25220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5618},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9562","reported_seq":9692,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.084422+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:16.084422+0000","last_peered":"2024-09-11T01:18:16.084422+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:16.084422+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9562,"log_dups_size":0,"ondisk_log_size":9562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39117766,"num_objects":9552,"num_object_clones":0,"num_object_copies":19104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6219,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5398},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632050398,"num_objects":154318,"num_object_clones":0,"num_object_copies":308636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":154318,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":113453,"num_write_kb":453812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4700,"num_bytes_recovered":19243491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280888832,"data_stored":1280737086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154390,"ondisk_log_size":154390,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692644,"kb_used_data":440212,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93679196,"statfs":{"total":96636764160,"available":95927496704,"internally_reserved":0,"allocated":450777088,"data_stored":450509783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518548,"kb_used_data":280004,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93853292,"statfs":{"total":96636764160,"available":96105771008,"internally_reserved":0,"allocated":286724096,"data_stored":286409358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":854768,"kb_used_data":530208,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93517072,"statfs":{"total":96636764160,"available":95761481728,"internally_reserved":0,"allocated":542932992,"data_stored":542661610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":542711808,"data_stored":542675678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":287469568,"data_stored":287390082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450707456,"data_stored":450671326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:18.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:18.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'9580", 'reported_seq': 9647, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.159207+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:16.159207+0000', 'last_peered': '2024-09-11T01:18:16.159207+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:16.159207+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 9580, 'log_dups_size': 0, 'ondisk_log_size': 9580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39239680, 'num_objects': 9580, 'num_object_clones': 0, 'num_object_copies': 19160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9580, 'num_write_kb': 38320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5457}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:18.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9487", 'reported_seq': 9565, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:15.183147+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:15.183147+0000', 'last_peered': '2024-09-11T01:18:15.183147+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:15.183147+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9487, 'log_dups_size': 0, 'ondisk_log_size': 9487, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38842368, 'num_objects': 9483, 'num_object_clones': 0, 'num_object_copies': 18966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7548, 'num_write_kb': 30192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5478}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:18.405 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:18.405 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9585", 'reported_seq': 9714, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.104225+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:16.104225+0000', 'last_peered': '2024-09-11T01:18:16.104225+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:16.104225+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9585, 'log_dups_size': 0, 'ondisk_log_size': 9585, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39240163, 'num_objects': 9581, 'num_object_clones': 0, 'num_object_copies': 19162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9581, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6073, 'num_write_kb': 24292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5466}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9853", 'reported_seq': 9980, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.053622+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:16.053622+0000', 'last_peered': '2024-09-11T01:18:16.053622+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:16.053622+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9853, 'log_dups_size': 0, 'ondisk_log_size': 9853, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40334278, 'num_objects': 9849, 'num_object_clones': 0, 'num_object_copies': 19698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9849, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6606, 'num_write_kb': 26424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5566}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'9749", 'reported_seq': 9873, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.084674+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:16.084674+0000', 'last_peered': '2024-09-11T01:18:16.084674+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:16.084674+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 9749, 'log_dups_size': 0, 'ondisk_log_size': 9749, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39899136, 'num_objects': 9741, 'num_object_clones': 0, 'num_object_copies': 19482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6113, 'num_write_kb': 24452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5636}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9480", 'reported_seq': 9569, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:15.107312+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:15.107312+0000', 'last_peered': '2024-09-11T01:18:15.107312+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:15.107312+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9480, 'log_dups_size': 0, 'ondisk_log_size': 9480, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38810083, 'num_objects': 9476, 'num_object_clones': 0, 'num_object_copies': 18952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9476, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7249, 'num_write_kb': 28996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5449}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'9750", 'reported_seq': 9838, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:15.166172+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:15.166172+0000', 'last_peered': '2024-09-11T01:18:15.166172+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:15.166172+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 9750, 'log_dups_size': 0, 'ondisk_log_size': 9750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39911424, 'num_objects': 9744, 'num_object_clones': 0, 'num_object_copies': 19488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7387, 'num_write_kb': 29548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5567}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:18.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'9677", 'reported_seq': 10951, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:16.110711+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:16.110711+0000', 'last_peered': '2024-09-11T01:18:16.110711+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:16.110711+0000', 'last_undegraded': '2024-09-11T01:18:16.110711+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 9677, 'log_dups_size': 0, 'ondisk_log_size': 9677, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39628800, 'num_objects': 9675, 'num_object_clones': 0, 'num_object_copies': 19350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3590, 'num_objects_unfound': 0, 'num_objects_dirty': 9675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6367, 'num_write_kb': 25468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 574, 'num_bytes_recovered': 2351104, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6085}, {'shards': '0,2', 'objects': 3590}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:18.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9815", 'reported_seq': 9944, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.134147+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:16.134147+0000', 'last_peered': '2024-09-11T01:18:16.134147+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:16.134147+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9815, 'log_dups_size': 0, 'ondisk_log_size': 9815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40169472, 'num_objects': 9807, 'num_object_clones': 0, 'num_object_copies': 19614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6208, 'num_write_kb': 24832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5615}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:18.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'9770", 'reported_seq': 9898, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.046167+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:16.046167+0000', 'last_peered': '2024-09-11T01:18:16.046167+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:16.046167+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 9770, 'log_dups_size': 0, 'ondisk_log_size': 9770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40001536, 'num_objects': 9766, 'num_object_clones': 0, 'num_object_copies': 19532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6305, 'num_write_kb': 25220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5618}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:18.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9562", 'reported_seq': 9692, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.084422+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:16.084422+0000', 'last_peered': '2024-09-11T01:18:16.084422+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:16.084422+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9562, 'log_dups_size': 0, 'ondisk_log_size': 9562, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39117766, 'num_objects': 9552, 'num_object_clones': 0, 'num_object_copies': 19104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9552, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6219, 'num_write_kb': 24876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5398}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.407 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:19.127 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:19.127 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/308640 objects degraded (13.395%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":299,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154320,"data_bytes":632050417,"bytes_used":2115543040,"bytes_avail":287794749440,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":308640,"degraded_ratio":0.13395217729393469,"misplaced_objects":3590,"misplaced_total":308640,"misplaced_ratio":0.011631674442716433,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":46403,"recovering_keys_per_sec":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"write_bytes_sec":1487630,"read_op_per_sec":0,"write_op_per_sec":363},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:19.155 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:19.155 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:19.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 223 20 155846 155826 2.72898 1.57812 0.058265 0.0286197 2024-09-11T01:18:19.754 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:19.756 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:19.781 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-11T01:18:18.486874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632050417,"num_objects":154320,"num_object_clones":0,"num_object_copies":308640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":154320,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":113455,"num_write_kb":453814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4701,"num_bytes_recovered":19243510,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154393,"ondisk_log_size":154393,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065960,"kb_used_data":1250424,"kb_used_omap":12,"kb_used_meta":815475,"kb_avail":281049560,"statfs":{"total":289910292480,"available":287794749440,"internally_reserved":0,"allocated":1280434176,"data_stored":1279580751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":835046407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,4,11,16,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17858560,"num_objects":4360,"num_object_clones":0,"num_object_copies":8720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-136,"num_objects_unfound":0,"num_objects_dirty":4360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4360,"num_write_kb":17440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":136,"num_bytes_recovered":557056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004700"},"pg_stats":[{"pgid":"2.f","version":"41'9580","reported_seq":9647,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.159207+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:16.159207+0000","last_peered":"2024-09-11T01:18:16.159207+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:16.159207+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9580,"log_dups_size":0,"ondisk_log_size":9580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39239680,"num_objects":9580,"num_object_clones":0,"num_object_copies":19160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9580,"num_write_kb":38320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5457},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9675","reported_seq":9747,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.195391+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:15.195391+0000","last_peered":"2024-09-11T01:18:15.195391+0000","last_clean":"2024-09-11T01:18:15.195391+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:15.195391+0000","last_undegraded":"2024-09-11T01:18:15.195391+0000","last_fullsized":"2024-09-11T01:18:15.195391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9675,"log_dups_size":0,"ondisk_log_size":9675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39605190,"num_objects":9671,"num_object_clones":0,"num_object_copies":19342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7773,"num_write_kb":31092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9487","reported_seq":9565,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.183147+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:15.183147+0000","last_peered":"2024-09-11T01:18:15.183147+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:15.183147+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9487,"log_dups_size":0,"ondisk_log_size":9487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38842368,"num_objects":9483,"num_object_clones":0,"num_object_copies":18966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7548,"num_write_kb":30192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5478}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9622","reported_seq":9662,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.131474+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:15.131474+0000","last_peered":"2024-09-11T01:18:15.131474+0000","last_clean":"2024-09-11T01:18:15.131474+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:15.131474+0000","last_undegraded":"2024-09-11T01:18:15.131474+0000","last_fullsized":"2024-09-11T01:18:15.131474+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9622,"log_dups_size":0,"ondisk_log_size":9622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39411712,"num_objects":9622,"num_object_clones":0,"num_object_copies":19244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9622,"num_write_kb":38488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9601","reported_seq":9717,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.119555+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:16.119555+0000","last_peered":"2024-09-11T01:18:16.119555+0000","last_clean":"2024-09-11T01:18:16.119555+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:16.119555+0000","last_undegraded":"2024-09-11T01:18:16.119555+0000","last_fullsized":"2024-09-11T01:18:16.119555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9601,"log_dups_size":0,"ondisk_log_size":9601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39297507,"num_objects":9595,"num_object_clones":0,"num_object_copies":19190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6399,"num_write_kb":25596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9585","reported_seq":9714,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.104225+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:16.104225+0000","last_peered":"2024-09-11T01:18:16.104225+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:16.104225+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9585,"log_dups_size":0,"ondisk_log_size":9585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6073,"num_write_kb":24292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5466},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9853","reported_seq":9980,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.053622+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:16.053622+0000","last_peered":"2024-09-11T01:18:16.053622+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:16.053622+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9853,"log_dups_size":0,"ondisk_log_size":9853,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6606,"num_write_kb":26424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5566},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9660","reported_seq":9774,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.104023+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:16.104023+0000","last_peered":"2024-09-11T01:18:16.104023+0000","last_clean":"2024-09-11T01:18:16.104023+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:16.104023+0000","last_undegraded":"2024-09-11T01:18:16.104023+0000","last_fullsized":"2024-09-11T01:18:16.104023+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9660,"log_dups_size":0,"ondisk_log_size":9660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559168,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6613,"num_write_kb":26452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9524","reported_seq":17862,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:13.737990+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:13.737990+0000","last_peered":"2024-09-11T01:18:13.737990+0000","last_clean":"2024-09-11T01:18:13.737990+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:13.737990+0000","last_undegraded":"2024-09-11T01:18:13.737990+0000","last_fullsized":"2024-09-11T01:18:13.737990+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9524,"log_dups_size":0,"ondisk_log_size":9524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38982115,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7391,"num_write_kb":29564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9749","reported_seq":9873,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.084674+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:16.084674+0000","last_peered":"2024-09-11T01:18:16.084674+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:16.084674+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39899136,"num_objects":9741,"num_object_clones":0,"num_object_copies":19482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6113,"num_write_kb":24452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892467+0000","last_peered":"2024-09-11T01:17:37.892467+0000","last_clean":"2024-09-11T01:17:37.892467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892467+0000","last_undegraded":"2024-09-11T01:17:37.892467+0000","last_fullsized":"2024-09-11T01:17:37.892467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9480","reported_seq":9569,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.107312+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:15.107312+0000","last_peered":"2024-09-11T01:18:15.107312+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:15.107312+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9480,"log_dups_size":0,"ondisk_log_size":9480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810083,"num_objects":9476,"num_object_clones":0,"num_object_copies":18952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7249,"num_write_kb":28996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5449}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9750","reported_seq":9838,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:15.166172+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:15.166172+0000","last_peered":"2024-09-11T01:18:15.166172+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:15.166172+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9750,"log_dups_size":0,"ondisk_log_size":9750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39911424,"num_objects":9744,"num_object_clones":0,"num_object_copies":19488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7387,"num_write_kb":29548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9677","reported_seq":10951,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:16.110711+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:16.110711+0000","last_peered":"2024-09-11T01:18:16.110711+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:16.110711+0000","last_undegraded":"2024-09-11T01:18:16.110711+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628800,"num_objects":9675,"num_object_clones":0,"num_object_copies":19350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":9675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6367,"num_write_kb":25468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2351104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6085},{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9815","reported_seq":9944,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.134147+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:16.134147+0000","last_peered":"2024-09-11T01:18:16.134147+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:16.134147+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9815,"log_dups_size":0,"ondisk_log_size":9815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40169472,"num_objects":9807,"num_object_clones":0,"num_object_copies":19614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6208,"num_write_kb":24832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9770","reported_seq":9898,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.046167+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:16.046167+0000","last_peered":"2024-09-11T01:18:16.046167+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:16.046167+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9770,"log_dups_size":0,"ondisk_log_size":9770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40001536,"num_objects":9766,"num_object_clones":0,"num_object_copies":19532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6305,"num_write_kb":25220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5618},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9562","reported_seq":9692,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.084422+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:16.084422+0000","last_peered":"2024-09-11T01:18:16.084422+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:16.084422+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9562,"log_dups_size":0,"ondisk_log_size":9562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39117766,"num_objects":9552,"num_object_clones":0,"num_object_copies":19104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6219,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5398},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632050398,"num_objects":154318,"num_object_clones":0,"num_object_copies":308636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":154318,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":113453,"num_write_kb":453812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4700,"num_bytes_recovered":19243491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1280888832,"data_stored":1280737086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154390,"ondisk_log_size":154390,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692644,"kb_used_data":440212,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93679196,"statfs":{"total":96636764160,"available":95927496704,"internally_reserved":0,"allocated":450777088,"data_stored":450509783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789988,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518548,"kb_used_data":280004,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93853292,"statfs":{"total":96636764160,"available":96105771008,"internally_reserved":0,"allocated":286724096,"data_stored":286409358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":854768,"kb_used_data":530208,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93517072,"statfs":{"total":96636764160,"available":95761481728,"internally_reserved":0,"allocated":542932992,"data_stored":542661610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":542711808,"data_stored":542675678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":287469568,"data_stored":287390082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450707456,"data_stored":450671326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:20.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 224 20 156275 156255 2.72428 1.67578 0.0322466 0.0286697 2024-09-11T01:18:21.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 225 20 156731 156711 2.72009 1.78125 0.0163119 0.0287143 2024-09-11T01:18:22.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 226 20 157192 157172 2.71602 1.80078 0.0168401 0.0287584 2024-09-11T01:18:22.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:23.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 227 20 157605 157585 2.71116 1.61328 0.120485 0.0288075 2024-09-11T01:18:23.374 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:23.377 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:23.401 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-11T01:18:22.487877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640787185,"num_objects":156453,"num_object_clones":0,"num_object_copies":312906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":156453,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":115588,"num_write_kb":462346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4766,"num_bytes_recovered":19509750,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156526,"ondisk_log_size":156526,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2097400,"kb_used_data":1267972,"kb_used_omap":12,"kb_used_meta":829363,"kb_avail":281018120,"statfs":{"total":289910292480,"available":287762554880,"internally_reserved":0,"allocated":1298403328,"data_stored":1297549903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":849267719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,8,6,13,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23277568,"num_objects":5683,"num_object_clones":0,"num_object_copies":11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-201,"num_objects_unfound":0,"num_objects_dirty":5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004800"},"pg_stats":[{"pgid":"2.f","version":"41'9729","reported_seq":9795,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.154219+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:21.154219+0000","last_peered":"2024-09-11T01:18:21.154219+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:21.154219+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9729,"log_dups_size":0,"ondisk_log_size":9729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39849984,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":38916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5606},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9826","reported_seq":9901,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.136816+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:20.136816+0000","last_peered":"2024-09-11T01:18:20.136816+0000","last_clean":"2024-09-11T01:18:20.136816+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:20.136816+0000","last_undegraded":"2024-09-11T01:18:20.136816+0000","last_fullsized":"2024-09-11T01:18:20.136816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40223686,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7924,"num_write_kb":31696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9619","reported_seq":9693,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.162115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:20.162115+0000","last_peered":"2024-09-11T01:18:20.162115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:20.162115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9619,"log_dups_size":0,"ondisk_log_size":9619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7680,"num_write_kb":30720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5610}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9739","reported_seq":9779,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.146742+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:20.146742+0000","last_peered":"2024-09-11T01:18:20.146742+0000","last_clean":"2024-09-11T01:18:20.146742+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:20.146742+0000","last_undegraded":"2024-09-11T01:18:20.146742+0000","last_fullsized":"2024-09-11T01:18:20.146742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9739,"num_write_kb":38956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9749","reported_seq":9865,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.050721+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:21.050721+0000","last_peered":"2024-09-11T01:18:21.050721+0000","last_clean":"2024-09-11T01:18:21.050721+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:21.050721+0000","last_undegraded":"2024-09-11T01:18:21.050721+0000","last_fullsized":"2024-09-11T01:18:21.050721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903715,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9709","reported_seq":9838,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.148839+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:21.148839+0000","last_peered":"2024-09-11T01:18:21.148839+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:21.148839+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748067,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6197,"num_write_kb":24788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5590},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9995","reported_seq":10123,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.201840+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:21.201840+0000","last_peered":"2024-09-11T01:18:21.201840+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:21.201840+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9995,"log_dups_size":0,"ondisk_log_size":9995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40915910,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9799","reported_seq":9913,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.196833+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:21.196833+0000","last_peered":"2024-09-11T01:18:21.196833+0000","last_clean":"2024-09-11T01:18:21.196833+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:21.196833+0000","last_undegraded":"2024-09-11T01:18:21.196833+0000","last_fullsized":"2024-09-11T01:18:21.196833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9659","reported_seq":17996,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:18.709473+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:18.709473+0000","last_peered":"2024-09-11T01:18:18.709473+0000","last_clean":"2024-09-11T01:18:18.709473+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:18.709473+0000","last_undegraded":"2024-09-11T01:18:18.709473+0000","last_fullsized":"2024-09-11T01:18:18.709473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9659,"log_dups_size":0,"ondisk_log_size":9659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39535075,"num_objects":9653,"num_object_clones":0,"num_object_copies":19306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7526,"num_write_kb":30104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9890","reported_seq":10014,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.174808+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:21.174808+0000","last_peered":"2024-09-11T01:18:21.174808+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:21.174808+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9890,"log_dups_size":0,"ondisk_log_size":9890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40476672,"num_objects":9882,"num_object_clones":0,"num_object_copies":19764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6254,"num_write_kb":25016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5777},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9609","reported_seq":9698,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.146957+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:20.146957+0000","last_peered":"2024-09-11T01:18:20.146957+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:20.146957+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338467,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7378,"num_write_kb":29512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9876","reported_seq":9964,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.192605+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:20.192605+0000","last_peered":"2024-09-11T01:18:20.192605+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:20.192605+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9876,"log_dups_size":0,"ondisk_log_size":9876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9798","reported_seq":11202,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:21.109019+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:21.109019+0000","last_peered":"2024-09-11T01:18:21.109019+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:21.109019+0000","last_undegraded":"2024-09-11T01:18:21.109019+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":639,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6271},{"shards":"0,2","objects":3525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9927","reported_seq":10056,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.148399+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:21.148399+0000","last_peered":"2024-09-11T01:18:21.148399+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:21.148399+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9927,"log_dups_size":0,"ondisk_log_size":9927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40628224,"num_objects":9919,"num_object_clones":0,"num_object_copies":19838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5727},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9901","reported_seq":10027,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.123586+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:21.123586+0000","last_peered":"2024-09-11T01:18:21.123586+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:21.123586+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538112,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6436,"num_write_kb":25744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5749},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9698","reported_seq":9827,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.192813+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:21.192813+0000","last_peered":"2024-09-11T01:18:21.192813+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:21.192813+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9698,"log_dups_size":0,"ondisk_log_size":9698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39674822,"num_objects":9688,"num_object_clones":0,"num_object_copies":19376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6355,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5534},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640787166,"num_objects":156451,"num_object_clones":0,"num_object_copies":312902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":156451,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":115586,"num_write_kb":462344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4765,"num_bytes_recovered":19509731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1298739200,"data_stored":1298587454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156523,"ondisk_log_size":156523,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701088,"kb_used_data":444496,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93670752,"statfs":{"total":96636764160,"available":95918850048,"internally_reserved":0,"allocated":455163904,"data_stored":454896599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530940,"kb_used_data":286824,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93840900,"statfs":{"total":96636764160,"available":96093081600,"internally_reserved":0,"allocated":293707776,"data_stored":293393038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865372,"kb_used_data":536652,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93506468,"statfs":{"total":96636764160,"available":95750623232,"internally_reserved":0,"allocated":549531648,"data_stored":549260266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":549261312,"data_stored":549225182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294412288,"data_stored":294332802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455065600,"data_stored":455029470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:23.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:23.971 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:23.973 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:24.001 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-11T01:18:22.487877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":640787185,"num_objects":156453,"num_object_clones":0,"num_object_copies":312906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":156453,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":115588,"num_write_kb":462346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4766,"num_bytes_recovered":19509750,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156526,"ondisk_log_size":156526,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2097400,"kb_used_data":1267972,"kb_used_omap":12,"kb_used_meta":829363,"kb_avail":281018120,"statfs":{"total":289910292480,"available":287762554880,"internally_reserved":0,"allocated":1298403328,"data_stored":1297549903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":849267719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,8,6,13,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23277568,"num_objects":5683,"num_object_clones":0,"num_object_copies":11366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-201,"num_objects_unfound":0,"num_objects_dirty":5683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5683,"num_write_kb":22732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":823296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004800"},"pg_stats":[{"pgid":"2.f","version":"41'9729","reported_seq":9795,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.154219+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:21.154219+0000","last_peered":"2024-09-11T01:18:21.154219+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:21.154219+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9729,"log_dups_size":0,"ondisk_log_size":9729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39849984,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":38916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5606},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9826","reported_seq":9901,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.136816+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:20.136816+0000","last_peered":"2024-09-11T01:18:20.136816+0000","last_clean":"2024-09-11T01:18:20.136816+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:20.136816+0000","last_undegraded":"2024-09-11T01:18:20.136816+0000","last_fullsized":"2024-09-11T01:18:20.136816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40223686,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7924,"num_write_kb":31696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9619","reported_seq":9693,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.162115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:20.162115+0000","last_peered":"2024-09-11T01:18:20.162115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:20.162115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9619,"log_dups_size":0,"ondisk_log_size":9619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7680,"num_write_kb":30720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5610}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9739","reported_seq":9779,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.146742+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:20.146742+0000","last_peered":"2024-09-11T01:18:20.146742+0000","last_clean":"2024-09-11T01:18:20.146742+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:20.146742+0000","last_undegraded":"2024-09-11T01:18:20.146742+0000","last_fullsized":"2024-09-11T01:18:20.146742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9739,"num_write_kb":38956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9749","reported_seq":9865,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.050721+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:21.050721+0000","last_peered":"2024-09-11T01:18:21.050721+0000","last_clean":"2024-09-11T01:18:21.050721+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:21.050721+0000","last_undegraded":"2024-09-11T01:18:21.050721+0000","last_fullsized":"2024-09-11T01:18:21.050721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903715,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9709","reported_seq":9838,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.148839+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:21.148839+0000","last_peered":"2024-09-11T01:18:21.148839+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:21.148839+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748067,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6197,"num_write_kb":24788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5590},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9995","reported_seq":10123,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.201840+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:21.201840+0000","last_peered":"2024-09-11T01:18:21.201840+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:21.201840+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9995,"log_dups_size":0,"ondisk_log_size":9995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40915910,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9799","reported_seq":9913,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.196833+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:21.196833+0000","last_peered":"2024-09-11T01:18:21.196833+0000","last_clean":"2024-09-11T01:18:21.196833+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:21.196833+0000","last_undegraded":"2024-09-11T01:18:21.196833+0000","last_fullsized":"2024-09-11T01:18:21.196833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9659","reported_seq":17996,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:18.709473+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:18.709473+0000","last_peered":"2024-09-11T01:18:18.709473+0000","last_clean":"2024-09-11T01:18:18.709473+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:18.709473+0000","last_undegraded":"2024-09-11T01:18:18.709473+0000","last_fullsized":"2024-09-11T01:18:18.709473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9659,"log_dups_size":0,"ondisk_log_size":9659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39535075,"num_objects":9653,"num_object_clones":0,"num_object_copies":19306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7526,"num_write_kb":30104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9890","reported_seq":10014,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.174808+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:21.174808+0000","last_peered":"2024-09-11T01:18:21.174808+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:21.174808+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9890,"log_dups_size":0,"ondisk_log_size":9890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40476672,"num_objects":9882,"num_object_clones":0,"num_object_copies":19764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6254,"num_write_kb":25016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5777},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9609","reported_seq":9698,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.146957+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:20.146957+0000","last_peered":"2024-09-11T01:18:20.146957+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:20.146957+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338467,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7378,"num_write_kb":29512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9876","reported_seq":9964,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.192605+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:20.192605+0000","last_peered":"2024-09-11T01:18:20.192605+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:20.192605+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9876,"log_dups_size":0,"ondisk_log_size":9876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9798","reported_seq":11202,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:21.109019+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:21.109019+0000","last_peered":"2024-09-11T01:18:21.109019+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:21.109019+0000","last_undegraded":"2024-09-11T01:18:21.109019+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":639,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6271},{"shards":"0,2","objects":3525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9927","reported_seq":10056,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.148399+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:21.148399+0000","last_peered":"2024-09-11T01:18:21.148399+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:21.148399+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9927,"log_dups_size":0,"ondisk_log_size":9927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40628224,"num_objects":9919,"num_object_clones":0,"num_object_copies":19838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5727},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9901","reported_seq":10027,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.123586+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:21.123586+0000","last_peered":"2024-09-11T01:18:21.123586+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:21.123586+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538112,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6436,"num_write_kb":25744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5749},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9698","reported_seq":9827,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.192813+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:21.192813+0000","last_peered":"2024-09-11T01:18:21.192813+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:21.192813+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9698,"log_dups_size":0,"ondisk_log_size":9698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39674822,"num_objects":9688,"num_object_clones":0,"num_object_copies":19376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6355,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5534},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":640787166,"num_objects":156451,"num_object_clones":0,"num_object_copies":312902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":156451,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":115586,"num_write_kb":462344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4765,"num_bytes_recovered":19509731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1298739200,"data_stored":1298587454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156523,"ondisk_log_size":156523,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701088,"kb_used_data":444496,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93670752,"statfs":{"total":96636764160,"available":95918850048,"internally_reserved":0,"allocated":455163904,"data_stored":454896599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530940,"kb_used_data":286824,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93840900,"statfs":{"total":96636764160,"available":96093081600,"internally_reserved":0,"allocated":293707776,"data_stored":293393038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865372,"kb_used_data":536652,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93506468,"statfs":{"total":96636764160,"available":95750623232,"internally_reserved":0,"allocated":549531648,"data_stored":549260266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":549261312,"data_stored":549225182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":294412288,"data_stored":294332802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455065600,"data_stored":455029470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'9729", 'reported_seq': 9795, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.154219+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:21.154219+0000', 'last_peered': '2024-09-11T01:18:21.154219+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:21.154219+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 9729, 'log_dups_size': 0, 'ondisk_log_size': 9729, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39849984, 'num_objects': 9729, 'num_object_clones': 0, 'num_object_copies': 19458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9729, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9729, 'num_write_kb': 38916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5606}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9619", 'reported_seq': 9693, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:20.162115+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:20.162115+0000', 'last_peered': '2024-09-11T01:18:20.162115+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:20.162115+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9619, 'log_dups_size': 0, 'ondisk_log_size': 9619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39383040, 'num_objects': 9615, 'num_object_clones': 0, 'num_object_copies': 19230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7680, 'num_write_kb': 30720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5610}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9709", 'reported_seq': 9838, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.148839+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:21.148839+0000', 'last_peered': '2024-09-11T01:18:21.148839+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:21.148839+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9709, 'log_dups_size': 0, 'ondisk_log_size': 9709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39748067, 'num_objects': 9705, 'num_object_clones': 0, 'num_object_copies': 19410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9705, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6197, 'num_write_kb': 24788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5590}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:24.002 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'9995", 'reported_seq': 10123, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.201840+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:21.201840+0000', 'last_peered': '2024-09-11T01:18:21.201840+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:21.201840+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 9995, 'log_dups_size': 0, 'ondisk_log_size': 9995, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40915910, 'num_objects': 9991, 'num_object_clones': 0, 'num_object_copies': 19982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9991, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6748, 'num_write_kb': 26992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5708}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'9890", 'reported_seq': 10014, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.174808+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:21.174808+0000', 'last_peered': '2024-09-11T01:18:21.174808+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:21.174808+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 9890, 'log_dups_size': 0, 'ondisk_log_size': 9890, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40476672, 'num_objects': 9882, 'num_object_clones': 0, 'num_object_copies': 19764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6254, 'num_write_kb': 25016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5777}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9609", 'reported_seq': 9698, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:20.146957+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:20.146957+0000', 'last_peered': '2024-09-11T01:18:20.146957+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:20.146957+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9609, 'log_dups_size': 0, 'ondisk_log_size': 9609, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39338467, 'num_objects': 9605, 'num_object_clones': 0, 'num_object_copies': 19210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9605, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7378, 'num_write_kb': 29512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5578}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'9876", 'reported_seq': 9964, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:20.192605+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:20.192605+0000', 'last_peered': '2024-09-11T01:18:20.192605+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:20.192605+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 9876, 'log_dups_size': 0, 'ondisk_log_size': 9876, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40427520, 'num_objects': 9870, 'num_object_clones': 0, 'num_object_copies': 19740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7513, 'num_write_kb': 30052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5693}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'9798", 'reported_seq': 11202, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:21.109019+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:21.109019+0000', 'last_peered': '2024-09-11T01:18:21.109019+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:21.109019+0000', 'last_undegraded': '2024-09-11T01:18:21.109019+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 9798, 'log_dups_size': 0, 'ondisk_log_size': 9798, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40124416, 'num_objects': 9796, 'num_object_clones': 0, 'num_object_copies': 19592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3525, 'num_objects_unfound': 0, 'num_objects_dirty': 9796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6488, 'num_write_kb': 25952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 639, 'num_bytes_recovered': 2617344, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6271}, {'shards': '0,2', 'objects': 3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'9927", 'reported_seq': 10056, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.148399+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:21.148399+0000', 'last_peered': '2024-09-11T01:18:21.148399+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:21.148399+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 9927, 'log_dups_size': 0, 'ondisk_log_size': 9927, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40628224, 'num_objects': 9919, 'num_object_clones': 0, 'num_object_copies': 19838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6320, 'num_write_kb': 25280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5727}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:24.003 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'9901", 'reported_seq': 10027, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.123586+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:21.123586+0000', 'last_peered': '2024-09-11T01:18:21.123586+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:21.123586+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 9901, 'log_dups_size': 0, 'ondisk_log_size': 9901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40538112, 'num_objects': 9897, 'num_object_clones': 0, 'num_object_copies': 19794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9897, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6436, 'num_write_kb': 25744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5749}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.004 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:24.004 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9698", 'reported_seq': 9827, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.192813+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:21.192813+0000', 'last_peered': '2024-09-11T01:18:21.192813+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:21.192813+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9698, 'log_dups_size': 0, 'ondisk_log_size': 9698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39674822, 'num_objects': 9688, 'num_object_clones': 0, 'num_object_copies': 19376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9688, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6355, 'num_write_kb': 25420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5534}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:24.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:24.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 228 20 158003 157983 2.70609 1.55469 0.0350714 0.0288634 2024-09-11T01:18:24.722 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:24.722 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/313140 objects degraded (13.203%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":305,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156570,"data_bytes":641266417,"bytes_used":2159022080,"bytes_avail":287751270400,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":313140,"degraded_ratio":0.13202720827744779,"misplaced_objects":3525,"misplaced_total":313140,"misplaced_ratio":0.011256945775052691,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":43332,"recovering_keys_per_sec":0,"num_objects_recovered":127,"num_bytes_recovered":520192,"num_keys_recovered":0,"write_bytes_sec":1519350,"read_op_per_sec":0,"write_op_per_sec":370},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:24.747 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:24.747 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:25.281 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:25.281 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:25.306 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-11T01:18:24.488365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641266417,"num_objects":156570,"num_object_clones":0,"num_object_copies":313140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":156570,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":115705,"num_write_kb":462814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4766,"num_bytes_recovered":19509750,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156643,"ondisk_log_size":156643,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2108420,"kb_used_data":1274836,"kb_used_omap":12,"kb_used_meta":833523,"kb_avail":281007100,"statfs":{"total":289910292480,"available":287751270400,"internally_reserved":0,"allocated":1305432064,"data_stored":1304578639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":853527559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,5,7,11,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18239488,"num_objects":4453,"num_object_clones":0,"num_object_copies":8906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-127,"num_objects_unfound":0,"num_objects_dirty":4453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4453,"num_write_kb":17812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":127,"num_bytes_recovered":520192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004792"},"pg_stats":[{"pgid":"2.f","version":"41'9729","reported_seq":9795,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.154219+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:21.154219+0000","last_peered":"2024-09-11T01:18:21.154219+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:21.154219+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9729,"log_dups_size":0,"ondisk_log_size":9729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39849984,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":38916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5606},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9826","reported_seq":9901,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.136816+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:20.136816+0000","last_peered":"2024-09-11T01:18:20.136816+0000","last_clean":"2024-09-11T01:18:20.136816+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:20.136816+0000","last_undegraded":"2024-09-11T01:18:20.136816+0000","last_fullsized":"2024-09-11T01:18:20.136816+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9826,"log_dups_size":0,"ondisk_log_size":9826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40223686,"num_objects":9822,"num_object_clones":0,"num_object_copies":19644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9822,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7924,"num_write_kb":31696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9619","reported_seq":9693,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.162115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:20.162115+0000","last_peered":"2024-09-11T01:18:20.162115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:20.162115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9619,"log_dups_size":0,"ondisk_log_size":9619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39383040,"num_objects":9615,"num_object_clones":0,"num_object_copies":19230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7680,"num_write_kb":30720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5610}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9739","reported_seq":9779,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.146742+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:20.146742+0000","last_peered":"2024-09-11T01:18:20.146742+0000","last_clean":"2024-09-11T01:18:20.146742+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:20.146742+0000","last_undegraded":"2024-09-11T01:18:20.146742+0000","last_fullsized":"2024-09-11T01:18:20.146742+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9739,"num_write_kb":38956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9749","reported_seq":9865,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.050721+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:21.050721+0000","last_peered":"2024-09-11T01:18:21.050721+0000","last_clean":"2024-09-11T01:18:21.050721+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:21.050721+0000","last_undegraded":"2024-09-11T01:18:21.050721+0000","last_fullsized":"2024-09-11T01:18:21.050721+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9749,"log_dups_size":0,"ondisk_log_size":9749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39903715,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6547,"num_write_kb":26188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9709","reported_seq":9838,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.148839+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:21.148839+0000","last_peered":"2024-09-11T01:18:21.148839+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:21.148839+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748067,"num_objects":9705,"num_object_clones":0,"num_object_copies":19410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9705,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6197,"num_write_kb":24788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5590},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'9995","reported_seq":10123,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.201840+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:21.201840+0000","last_peered":"2024-09-11T01:18:21.201840+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:21.201840+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":9995,"log_dups_size":0,"ondisk_log_size":9995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40915910,"num_objects":9991,"num_object_clones":0,"num_object_copies":19982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9991,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6748,"num_write_kb":26992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9799","reported_seq":9913,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.196833+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:21.196833+0000","last_peered":"2024-09-11T01:18:21.196833+0000","last_clean":"2024-09-11T01:18:21.196833+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:21.196833+0000","last_undegraded":"2024-09-11T01:18:21.196833+0000","last_fullsized":"2024-09-11T01:18:21.196833+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6752,"num_write_kb":27008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9776","reported_seq":18114,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:23.798639+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:23.798639+0000","last_peered":"2024-09-11T01:18:23.798639+0000","last_clean":"2024-09-11T01:18:23.798639+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:23.798639+0000","last_undegraded":"2024-09-11T01:18:23.798639+0000","last_fullsized":"2024-09-11T01:18:23.798639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40014307,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7643,"num_write_kb":30572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'9890","reported_seq":10014,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.174808+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:21.174808+0000","last_peered":"2024-09-11T01:18:21.174808+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:21.174808+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":9890,"log_dups_size":0,"ondisk_log_size":9890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40476672,"num_objects":9882,"num_object_clones":0,"num_object_copies":19764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6254,"num_write_kb":25016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5777},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9609","reported_seq":9698,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.146957+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:20.146957+0000","last_peered":"2024-09-11T01:18:20.146957+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:20.146957+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9609,"log_dups_size":0,"ondisk_log_size":9609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39338467,"num_objects":9605,"num_object_clones":0,"num_object_copies":19210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9605,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7378,"num_write_kb":29512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'9876","reported_seq":9964,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:20.192605+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:20.192605+0000","last_peered":"2024-09-11T01:18:20.192605+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:20.192605+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":9876,"log_dups_size":0,"ondisk_log_size":9876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40427520,"num_objects":9870,"num_object_clones":0,"num_object_copies":19740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5693}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9798","reported_seq":11202,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:21.109019+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:21.109019+0000","last_peered":"2024-09-11T01:18:21.109019+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:21.109019+0000","last_undegraded":"2024-09-11T01:18:21.109019+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9798,"log_dups_size":0,"ondisk_log_size":9798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6488,"num_write_kb":25952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":639,"num_bytes_recovered":2617344,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6271},{"shards":"0,2","objects":3525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'9927","reported_seq":10056,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.148399+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:21.148399+0000","last_peered":"2024-09-11T01:18:21.148399+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:21.148399+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":9927,"log_dups_size":0,"ondisk_log_size":9927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40628224,"num_objects":9919,"num_object_clones":0,"num_object_copies":19838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":25280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5727},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'9901","reported_seq":10027,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.123586+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:21.123586+0000","last_peered":"2024-09-11T01:18:21.123586+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:21.123586+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40538112,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6436,"num_write_kb":25744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5749},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9698","reported_seq":9827,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.192813+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:21.192813+0000","last_peered":"2024-09-11T01:18:21.192813+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:21.192813+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9698,"log_dups_size":0,"ondisk_log_size":9698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39674822,"num_objects":9688,"num_object_clones":0,"num_object_copies":19376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6355,"num_write_kb":25420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5534},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641266398,"num_objects":156568,"num_object_clones":0,"num_object_copies":313136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3525,"num_objects_unfound":0,"num_objects_dirty":156568,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":115703,"num_write_kb":462812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4765,"num_bytes_recovered":19509731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1305784320,"data_stored":1305632574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156640,"ondisk_log_size":156640,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701088,"kb_used_data":444496,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93670752,"statfs":{"total":96636764160,"available":95918850048,"internally_reserved":0,"allocated":455163904,"data_stored":454896599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541960,"kb_used_data":293688,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93829880,"statfs":{"total":96636764160,"available":96081797120,"internally_reserved":0,"allocated":300736512,"data_stored":300421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865372,"kb_used_data":536652,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93506468,"statfs":{"total":96636764160,"available":95750623232,"internally_reserved":0,"allocated":549531648,"data_stored":549260266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,6,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":549261312,"data_stored":549225182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301457408,"data_stored":301377922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455065600,"data_stored":455029470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:25.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 229 20 158406 158386 2.70115 1.57422 0.0305652 0.0289161 2024-09-11T01:18:26.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 230 20 158867 158847 2.69723 1.80078 0.0272978 0.0289572 2024-09-11T01:18:27.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 231 20 159323 159303 2.69327 1.78125 0.014421 0.0290008 2024-09-11T01:18:28.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:28.360 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 232 20 159773 159753 2.68924 1.75781 0.0354205 0.0290417 2024-09-11T01:18:28.888 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:28.888 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:28.918 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-11T01:18:28.489361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649597681,"num_objects":158604,"num_object_clones":0,"num_object_copies":317208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":158604,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":117739,"num_write_kb":470950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4831,"num_bytes_recovered":19775990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158577,"ondisk_log_size":158577,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2098336,"kb_used_data":1285548,"kb_used_omap":12,"kb_used_meta":812723,"kb_avail":281017184,"statfs":{"total":289910292480,"available":287761596416,"internally_reserved":0,"allocated":1316401152,"data_stored":1315547727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":832228359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,7,8,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17547264,"num_objects":4284,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-130,"num_objects_unfound":0,"num_objects_dirty":4284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4284,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003006"},"pg_stats":[{"pgid":"2.f","version":"41'9889","reported_seq":9957,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.247327+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:26.247327+0000","last_peered":"2024-09-11T01:18:26.247327+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:26.247327+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9889,"log_dups_size":0,"ondisk_log_size":9889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40505344,"num_objects":9889,"num_object_clones":0,"num_object_copies":19778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5766},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9962","reported_seq":10037,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:25.215902+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:25.215902+0000","last_peered":"2024-09-11T01:18:25.215902+0000","last_clean":"2024-09-11T01:18:25.215902+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:25.215902+0000","last_undegraded":"2024-09-11T01:18:25.215902+0000","last_fullsized":"2024-09-11T01:18:25.215902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9962,"log_dups_size":0,"ondisk_log_size":9962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40780742,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8060,"num_write_kb":32240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9763","reported_seq":9841,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:25.112733+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:25.112733+0000","last_peered":"2024-09-11T01:18:25.112733+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:25.112733+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9763,"log_dups_size":0,"ondisk_log_size":9763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39972864,"num_objects":9759,"num_object_clones":0,"num_object_copies":19518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7824,"num_write_kb":31296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9880","reported_seq":9920,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:25.257336+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:25.257336+0000","last_peered":"2024-09-11T01:18:25.257336+0000","last_clean":"2024-09-11T01:18:25.257336+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:25.257336+0000","last_undegraded":"2024-09-11T01:18:25.257336+0000","last_fullsized":"2024-09-11T01:18:25.257336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9880,"log_dups_size":0,"ondisk_log_size":9880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40468480,"num_objects":9880,"num_object_clones":0,"num_object_copies":19760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":39520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9865","reported_seq":9981,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.259956+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:26.259956+0000","last_peered":"2024-09-11T01:18:26.259956+0000","last_clean":"2024-09-11T01:18:26.259956+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:26.259956+0000","last_undegraded":"2024-09-11T01:18:26.259956+0000","last_fullsized":"2024-09-11T01:18:26.259956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9865,"log_dups_size":0,"ondisk_log_size":9865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378851,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6663,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9839","reported_seq":9968,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.192967+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:26.192967+0000","last_peered":"2024-09-11T01:18:26.192967+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:26.192967+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40280547,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6327,"num_write_kb":25308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5720},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10140","reported_seq":10268,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.233955+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:26.233955+0000","last_peered":"2024-09-11T01:18:26.233955+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:26.233955+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41509830,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6893,"num_write_kb":27572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5853},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9919","reported_seq":10033,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.258109+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:26.258109+0000","last_peered":"2024-09-11T01:18:26.258109+0000","last_clean":"2024-09-11T01:18:26.258109+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:26.258109+0000","last_undegraded":"2024-09-11T01:18:26.258109+0000","last_fullsized":"2024-09-11T01:18:26.258109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9919,"log_dups_size":0,"ondisk_log_size":9919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9776","reported_seq":18114,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:23.798639+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:23.798639+0000","last_peered":"2024-09-11T01:18:23.798639+0000","last_clean":"2024-09-11T01:18:23.798639+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:23.798639+0000","last_undegraded":"2024-09-11T01:18:23.798639+0000","last_fullsized":"2024-09-11T01:18:23.798639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40014307,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7643,"num_write_kb":30572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10037","reported_seq":10161,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.161756+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:26.161756+0000","last_peered":"2024-09-11T01:18:26.161756+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:26.161756+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078784,"num_objects":10029,"num_object_clones":0,"num_object_copies":20058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5924},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9759","reported_seq":9847,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:25.216623+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:25.216623+0000","last_peered":"2024-09-11T01:18:25.216623+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:25.216623+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9759,"log_dups_size":0,"ondisk_log_size":9759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952867,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7528,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5728}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10013","reported_seq":10099,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:25.174900+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:25.174900+0000","last_peered":"2024-09-11T01:18:25.174900+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:25.174900+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7650,"num_write_kb":30600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9928","reported_seq":11462,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:26.262112+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:26.262112+0000","last_peered":"2024-09-11T01:18:26.262112+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:26.262112+0000","last_undegraded":"2024-09-11T01:18:26.262112+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9928,"log_dups_size":0,"ondisk_log_size":9928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":704,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":3460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10041","reported_seq":10169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.247534+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:26.247534+0000","last_peered":"2024-09-11T01:18:26.247534+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:26.247534+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10041","reported_seq":10168,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.218233+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:26.218233+0000","last_peered":"2024-09-11T01:18:26.218233+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:26.218233+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41111552,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6576,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5889},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9822","reported_seq":9951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.236587+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:26.236587+0000","last_peered":"2024-09-11T01:18:26.236587+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:26.236587+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9822,"log_dups_size":0,"ondisk_log_size":9822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40182726,"num_objects":9812,"num_object_clones":0,"num_object_copies":19624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6479,"num_write_kb":25916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649597662,"num_objects":158602,"num_object_clones":0,"num_object_copies":317204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":158602,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":117737,"num_write_kb":470948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4830,"num_bytes_recovered":19775971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1316835328,"data_stored":1316683582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158574,"ondisk_log_size":158574,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692940,"kb_used_data":448824,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93678900,"statfs":{"total":96636764160,"available":95927193600,"internally_reserved":0,"allocated":459595776,"data_stored":459328471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541960,"kb_used_data":293688,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93829880,"statfs":{"total":96636764160,"available":96081797120,"internally_reserved":0,"allocated":300736512,"data_stored":300421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863436,"kb_used_data":543036,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93508404,"statfs":{"total":96636764160,"available":95752605696,"internally_reserved":0,"allocated":556068864,"data_stored":555797482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555917312,"data_stored":555881182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301457408,"data_stored":301377922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459460608,"data_stored":459424478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:28.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:29.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 233 20 160176 160156 2.68445 1.57422 0.0326666 0.029095 2024-09-11T01:18:29.513 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:29.515 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:29.551 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-11T01:18:28.489361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649597681,"num_objects":158604,"num_object_clones":0,"num_object_copies":317208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":158604,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":117739,"num_write_kb":470950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4831,"num_bytes_recovered":19775990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158577,"ondisk_log_size":158577,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2098336,"kb_used_data":1285548,"kb_used_omap":12,"kb_used_meta":812723,"kb_avail":281017184,"statfs":{"total":289910292480,"available":287761596416,"internally_reserved":0,"allocated":1316401152,"data_stored":1315547727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":832228359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,7,8,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17547264,"num_objects":4284,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-130,"num_objects_unfound":0,"num_objects_dirty":4284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4284,"num_write_kb":17136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003006"},"pg_stats":[{"pgid":"2.f","version":"41'9889","reported_seq":9957,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.247327+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:26.247327+0000","last_peered":"2024-09-11T01:18:26.247327+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:26.247327+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9889,"log_dups_size":0,"ondisk_log_size":9889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40505344,"num_objects":9889,"num_object_clones":0,"num_object_copies":19778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5766},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'9962","reported_seq":10037,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:25.215902+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:25.215902+0000","last_peered":"2024-09-11T01:18:25.215902+0000","last_clean":"2024-09-11T01:18:25.215902+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:25.215902+0000","last_undegraded":"2024-09-11T01:18:25.215902+0000","last_fullsized":"2024-09-11T01:18:25.215902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":9962,"log_dups_size":0,"ondisk_log_size":9962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40780742,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8060,"num_write_kb":32240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9763","reported_seq":9841,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:25.112733+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:25.112733+0000","last_peered":"2024-09-11T01:18:25.112733+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:25.112733+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9763,"log_dups_size":0,"ondisk_log_size":9763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39972864,"num_objects":9759,"num_object_clones":0,"num_object_copies":19518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7824,"num_write_kb":31296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5754}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'9880","reported_seq":9920,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:25.257336+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:25.257336+0000","last_peered":"2024-09-11T01:18:25.257336+0000","last_clean":"2024-09-11T01:18:25.257336+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:25.257336+0000","last_undegraded":"2024-09-11T01:18:25.257336+0000","last_fullsized":"2024-09-11T01:18:25.257336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9880,"log_dups_size":0,"ondisk_log_size":9880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40468480,"num_objects":9880,"num_object_clones":0,"num_object_copies":19760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":39520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9865","reported_seq":9981,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.259956+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:26.259956+0000","last_peered":"2024-09-11T01:18:26.259956+0000","last_clean":"2024-09-11T01:18:26.259956+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:26.259956+0000","last_undegraded":"2024-09-11T01:18:26.259956+0000","last_fullsized":"2024-09-11T01:18:26.259956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9865,"log_dups_size":0,"ondisk_log_size":9865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378851,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6663,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9839","reported_seq":9968,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.192967+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:26.192967+0000","last_peered":"2024-09-11T01:18:26.192967+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:26.192967+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40280547,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6327,"num_write_kb":25308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5720},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10140","reported_seq":10268,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.233955+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:26.233955+0000","last_peered":"2024-09-11T01:18:26.233955+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:26.233955+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41509830,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6893,"num_write_kb":27572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5853},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9919","reported_seq":10033,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.258109+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:26.258109+0000","last_peered":"2024-09-11T01:18:26.258109+0000","last_clean":"2024-09-11T01:18:26.258109+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:26.258109+0000","last_undegraded":"2024-09-11T01:18:26.258109+0000","last_fullsized":"2024-09-11T01:18:26.258109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9919,"log_dups_size":0,"ondisk_log_size":9919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9776","reported_seq":18114,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:23.798639+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:23.798639+0000","last_peered":"2024-09-11T01:18:23.798639+0000","last_clean":"2024-09-11T01:18:23.798639+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:23.798639+0000","last_undegraded":"2024-09-11T01:18:23.798639+0000","last_fullsized":"2024-09-11T01:18:23.798639+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40014307,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7643,"num_write_kb":30572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10037","reported_seq":10161,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.161756+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:26.161756+0000","last_peered":"2024-09-11T01:18:26.161756+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:26.161756+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078784,"num_objects":10029,"num_object_clones":0,"num_object_copies":20058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5924},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9759","reported_seq":9847,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:25.216623+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:25.216623+0000","last_peered":"2024-09-11T01:18:25.216623+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:25.216623+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9759,"log_dups_size":0,"ondisk_log_size":9759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952867,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7528,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5728}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10013","reported_seq":10099,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:25.174900+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:25.174900+0000","last_peered":"2024-09-11T01:18:25.174900+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:25.174900+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40988672,"num_objects":10007,"num_object_clones":0,"num_object_copies":20014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7650,"num_write_kb":30600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9928","reported_seq":11462,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:26.262112+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:26.262112+0000","last_peered":"2024-09-11T01:18:26.262112+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:26.262112+0000","last_undegraded":"2024-09-11T01:18:26.262112+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9928,"log_dups_size":0,"ondisk_log_size":9928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":704,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":3460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10041","reported_seq":10169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.247534+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:26.247534+0000","last_peered":"2024-09-11T01:18:26.247534+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:26.247534+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10041","reported_seq":10168,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.218233+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:26.218233+0000","last_peered":"2024-09-11T01:18:26.218233+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:26.218233+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41111552,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6576,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5889},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9822","reported_seq":9951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.236587+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:26.236587+0000","last_peered":"2024-09-11T01:18:26.236587+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:26.236587+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9822,"log_dups_size":0,"ondisk_log_size":9822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40182726,"num_objects":9812,"num_object_clones":0,"num_object_copies":19624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6479,"num_write_kb":25916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":649597662,"num_objects":158602,"num_object_clones":0,"num_object_copies":317204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":158602,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":117737,"num_write_kb":470948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4830,"num_bytes_recovered":19775971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1316835328,"data_stored":1316683582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158574,"ondisk_log_size":158574,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692940,"kb_used_data":448824,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93678900,"statfs":{"total":96636764160,"available":95927193600,"internally_reserved":0,"allocated":459595776,"data_stored":459328471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":249949870},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541960,"kb_used_data":293688,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93829880,"statfs":{"total":96636764160,"available":96081797120,"internally_reserved":0,"allocated":300736512,"data_stored":300421774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863436,"kb_used_data":543036,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93508404,"statfs":{"total":96636764160,"available":95752605696,"internally_reserved":0,"allocated":556068864,"data_stored":555797482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555917312,"data_stored":555881182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301457408,"data_stored":301377922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459460608,"data_stored":459424478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'9889", 'reported_seq': 9957, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.247327+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:26.247327+0000', 'last_peered': '2024-09-11T01:18:26.247327+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:26.247327+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 9889, 'log_dups_size': 0, 'ondisk_log_size': 9889, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40505344, 'num_objects': 9889, 'num_object_clones': 0, 'num_object_copies': 19778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9889, 'num_write_kb': 39556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5766}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9763", 'reported_seq': 9841, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:25.112733+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:25.112733+0000', 'last_peered': '2024-09-11T01:18:25.112733+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:25.112733+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9763, 'log_dups_size': 0, 'ondisk_log_size': 9763, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39972864, 'num_objects': 9759, 'num_object_clones': 0, 'num_object_copies': 19518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7824, 'num_write_kb': 31296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5754}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9839", 'reported_seq': 9968, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.192967+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:26.192967+0000', 'last_peered': '2024-09-11T01:18:26.192967+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:26.192967+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9839, 'log_dups_size': 0, 'ondisk_log_size': 9839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40280547, 'num_objects': 9835, 'num_object_clones': 0, 'num_object_copies': 19670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9835, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6327, 'num_write_kb': 25308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5720}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'10140", 'reported_seq': 10268, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.233955+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:26.233955+0000', 'last_peered': '2024-09-11T01:18:26.233955+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:26.233955+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41509830, 'num_objects': 10136, 'num_object_clones': 0, 'num_object_copies': 20272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10136, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 6893, 'num_write_kb': 27572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5853}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:29.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10037", 'reported_seq': 10161, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.161756+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:26.161756+0000', 'last_peered': '2024-09-11T01:18:26.161756+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:26.161756+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41078784, 'num_objects': 10029, 'num_object_clones': 0, 'num_object_copies': 20058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6401, 'num_write_kb': 25604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5924}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9759", 'reported_seq': 9847, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:25.216623+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:25.216623+0000', 'last_peered': '2024-09-11T01:18:25.216623+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:25.216623+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9759, 'log_dups_size': 0, 'ondisk_log_size': 9759, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39952867, 'num_objects': 9755, 'num_object_clones': 0, 'num_object_copies': 19510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9755, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7528, 'num_write_kb': 30112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5728}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10013", 'reported_seq': 10099, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:25.174900+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:25.174900+0000', 'last_peered': '2024-09-11T01:18:25.174900+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:25.174900+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40988672, 'num_objects': 10007, 'num_object_clones': 0, 'num_object_copies': 20014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7650, 'num_write_kb': 30600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5830}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'9928", 'reported_seq': 11462, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:26.262112+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:26.262112+0000', 'last_peered': '2024-09-11T01:18:26.262112+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:26.262112+0000', 'last_undegraded': '2024-09-11T01:18:26.262112+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 9928, 'log_dups_size': 0, 'ondisk_log_size': 9928, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40656896, 'num_objects': 9926, 'num_object_clones': 0, 'num_object_copies': 19852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3460, 'num_objects_unfound': 0, 'num_objects_dirty': 9926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6618, 'num_write_kb': 26472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 704, 'num_bytes_recovered': 2883584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6466}, {'shards': '0,2', 'objects': 3460}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'10041", 'reported_seq': 10169, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.247534+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:26.247534+0000', 'last_peered': '2024-09-11T01:18:26.247534+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:26.247534+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41095168, 'num_objects': 10033, 'num_object_clones': 0, 'num_object_copies': 20066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6434, 'num_write_kb': 25736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5841}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10041", 'reported_seq': 10168, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.218233+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:26.218233+0000', 'last_peered': '2024-09-11T01:18:26.218233+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:26.218233+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41111552, 'num_objects': 10037, 'num_object_clones': 0, 'num_object_copies': 20074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6576, 'num_write_kb': 26304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5889}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:29.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9822", 'reported_seq': 9951, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.236587+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:26.236587+0000', 'last_peered': '2024-09-11T01:18:26.236587+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:26.236587+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9822, 'log_dups_size': 0, 'ondisk_log_size': 9822, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40182726, 'num_objects': 9812, 'num_object_clones': 0, 'num_object_copies': 19624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9812, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6479, 'num_write_kb': 25916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5658}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:29.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:30.269 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:30.269 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/317208 objects degraded (13.033%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":310,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158604,"data_bytes":649597681,"bytes_used":2148696064,"bytes_avail":287761596416,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":317208,"degraded_ratio":0.1303340394945903,"misplaced_objects":3460,"misplaced_total":317208,"misplaced_ratio":0.01090766941565156,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":44362,"recovering_keys_per_sec":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"write_bytes_sec":1461905,"read_op_per_sec":0,"write_op_per_sec":356},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:30.297 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:30.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:30.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 234 20 160575 160555 2.67964 1.55859 0.0264394 0.0291474 2024-09-11T01:18:30.882 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:30.885 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:30.911 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-11T01:18:30.489817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652989169,"num_objects":159432,"num_object_clones":0,"num_object_copies":318864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":159432,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":118567,"num_write_kb":474262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4831,"num_bytes_recovered":19775990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159305,"ondisk_log_size":159305,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117888,"kb_used_data":1296784,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":280997632,"statfs":{"total":289910292480,"available":287741575168,"internally_reserved":0,"allocated":1327906816,"data_stored":1327053391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":840748039},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,4,10,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20938752,"num_objects":5112,"num_object_clones":0,"num_object_copies":10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-130,"num_objects_unfound":0,"num_objects_dirty":5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5112,"num_write_kb":20448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002942"},"pg_stats":[{"pgid":"2.f","version":"41'9889","reported_seq":9957,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.247327+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:26.247327+0000","last_peered":"2024-09-11T01:18:26.247327+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:26.247327+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":9889,"log_dups_size":0,"ondisk_log_size":9889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40505344,"num_objects":9889,"num_object_clones":0,"num_object_copies":19778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":39556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5766},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10093","reported_seq":10168,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.229901+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:30.229901+0000","last_peered":"2024-09-11T01:18:30.229901+0000","last_clean":"2024-09-11T01:18:30.229901+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:30.229901+0000","last_undegraded":"2024-09-11T01:18:30.229901+0000","last_fullsized":"2024-09-11T01:18:30.229901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41317318,"num_objects":10089,"num_object_clones":0,"num_object_copies":20178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10089,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8191,"num_write_kb":32764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9885","reported_seq":9963,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.224624+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:30.224624+0000","last_peered":"2024-09-11T01:18:30.224624+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:30.224624+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9885,"log_dups_size":0,"ondisk_log_size":9885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7946,"num_write_kb":31784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10019","reported_seq":10059,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.170025+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:30.170025+0000","last_peered":"2024-09-11T01:18:30.170025+0000","last_clean":"2024-09-11T01:18:30.170025+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:30.170025+0000","last_undegraded":"2024-09-11T01:18:30.170025+0000","last_fullsized":"2024-09-11T01:18:30.170025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41037824,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10019,"num_write_kb":40076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'9865","reported_seq":9981,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.259956+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:26.259956+0000","last_peered":"2024-09-11T01:18:26.259956+0000","last_clean":"2024-09-11T01:18:26.259956+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:26.259956+0000","last_undegraded":"2024-09-11T01:18:26.259956+0000","last_fullsized":"2024-09-11T01:18:26.259956+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":9865,"log_dups_size":0,"ondisk_log_size":9865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40378851,"num_objects":9859,"num_object_clones":0,"num_object_copies":19718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6663,"num_write_kb":26652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9839","reported_seq":9968,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.192967+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:26.192967+0000","last_peered":"2024-09-11T01:18:26.192967+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:26.192967+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9839,"log_dups_size":0,"ondisk_log_size":9839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40280547,"num_objects":9835,"num_object_clones":0,"num_object_copies":19670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6327,"num_write_kb":25308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5720},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10140","reported_seq":10268,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.233955+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:26.233955+0000","last_peered":"2024-09-11T01:18:26.233955+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:26.233955+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41509830,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6893,"num_write_kb":27572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5853},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'9919","reported_seq":10033,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.258109+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:26.258109+0000","last_peered":"2024-09-11T01:18:26.258109+0000","last_clean":"2024-09-11T01:18:26.258109+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:26.258109+0000","last_undegraded":"2024-09-11T01:18:26.258109+0000","last_fullsized":"2024-09-11T01:18:26.258109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":9919,"log_dups_size":0,"ondisk_log_size":9919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40620032,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6872,"num_write_kb":27488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'9928","reported_seq":18266,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:28.778480+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:28.778480+0000","last_peered":"2024-09-11T01:18:28.778480+0000","last_clean":"2024-09-11T01:18:28.778480+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:28.778480+0000","last_undegraded":"2024-09-11T01:18:28.778480+0000","last_fullsized":"2024-09-11T01:18:28.778480+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":9928,"log_dups_size":0,"ondisk_log_size":9928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40636899,"num_objects":9922,"num_object_clones":0,"num_object_copies":19844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7795,"num_write_kb":31180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10037","reported_seq":10161,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.161756+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:26.161756+0000","last_peered":"2024-09-11T01:18:26.161756+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:26.161756+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41078784,"num_objects":10029,"num_object_clones":0,"num_object_copies":20058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6401,"num_write_kb":25604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5924},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9904","reported_seq":9993,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.173071+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:30.173071+0000","last_peered":"2024-09-11T01:18:30.173071+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:30.173071+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9904,"log_dups_size":0,"ondisk_log_size":9904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546787,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7673,"num_write_kb":30692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10152","reported_seq":10237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.189503+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:30.189503+0000","last_peered":"2024-09-11T01:18:30.189503+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:30.189503+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41558016,"num_objects":10146,"num_object_clones":0,"num_object_copies":20292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7789,"num_write_kb":31156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'9928","reported_seq":11462,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:26.262112+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:26.262112+0000","last_peered":"2024-09-11T01:18:26.262112+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:26.262112+0000","last_undegraded":"2024-09-11T01:18:26.262112+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":9928,"log_dups_size":0,"ondisk_log_size":9928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6618,"num_write_kb":26472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":704,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":3460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10041","reported_seq":10169,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.247534+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:26.247534+0000","last_peered":"2024-09-11T01:18:26.247534+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:26.247534+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6434,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5841},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10041","reported_seq":10168,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.218233+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:26.218233+0000","last_peered":"2024-09-11T01:18:26.218233+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:26.218233+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41111552,"num_objects":10037,"num_object_clones":0,"num_object_copies":20074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6576,"num_write_kb":26304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5889},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9822","reported_seq":9951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.236587+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:26.236587+0000","last_peered":"2024-09-11T01:18:26.236587+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:26.236587+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9822,"log_dups_size":0,"ondisk_log_size":9822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40182726,"num_objects":9812,"num_object_clones":0,"num_object_copies":19624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9812,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6479,"num_write_kb":25916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5658},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":652989150,"num_objects":159430,"num_object_clones":0,"num_object_copies":318860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3460,"num_objects_unfound":0,"num_objects_dirty":159430,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":118565,"num_write_kb":474260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4830,"num_bytes_recovered":19775971,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328328704,"data_stored":1328176958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159302,"ondisk_log_size":159302,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701588,"kb_used_data":453316,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93670252,"statfs":{"total":96636764160,"available":95918338048,"internally_reserved":0,"allocated":464195584,"data_stored":463928279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789991,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552864,"kb_used_data":300432,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93818976,"statfs":{"total":96636764160,"available":96070631424,"internally_reserved":0,"allocated":307642368,"data_stored":307327630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863436,"kb_used_data":543036,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93508404,"statfs":{"total":96636764160,"available":95752605696,"internally_reserved":0,"allocated":556068864,"data_stored":555797482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555917312,"data_stored":555881182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308416512,"data_stored":308337026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463994880,"data_stored":463958750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:31.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 235 20 160997 160977 2.67525 1.64844 0.0469087 0.0291955 2024-09-11T01:18:32.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 236 20 161459 161439 2.67156 1.80469 0.0249494 0.0292363 2024-09-11T01:18:33.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 237 20 161832 161812 2.66644 1.45703 0.0722874 0.0292915 2024-09-11T01:18:33.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:34.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 238 19 162255 162236 2.66219 1.65625 0.032157 0.0293379 2024-09-11T01:18:34.524 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:34.526 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:34.551 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-11T01:18:34.490841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659104497,"num_objects":160925,"num_object_clones":0,"num_object_copies":321850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3395,"num_objects_unfound":0,"num_objects_dirty":160925,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":120060,"num_write_kb":480234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4896,"num_bytes_recovered":20042230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160398,"ondisk_log_size":160398,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127648,"kb_used_data":1310704,"kb_used_omap":12,"kb_used_meta":816883,"kb_avail":280987872,"statfs":{"total":289910292480,"available":287731580928,"internally_reserved":0,"allocated":1342160896,"data_stored":1341307471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":836488199},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,7,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18317312,"num_objects":4472,"num_object_clones":0,"num_object_copies":8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-130,"num_objects_unfound":0,"num_objects_dirty":4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4472,"num_write_kb":17888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002963"},"pg_stats":[{"pgid":"2.f","version":"41'10006","reported_seq":10074,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.260453+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:31.260453+0000","last_peered":"2024-09-11T01:18:31.260453+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:31.260453+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40984576,"num_objects":10006,"num_object_clones":0,"num_object_copies":20012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10006,"num_write_kb":40024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10093","reported_seq":10168,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.229901+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:30.229901+0000","last_peered":"2024-09-11T01:18:30.229901+0000","last_clean":"2024-09-11T01:18:30.229901+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:30.229901+0000","last_undegraded":"2024-09-11T01:18:30.229901+0000","last_fullsized":"2024-09-11T01:18:30.229901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41317318,"num_objects":10089,"num_object_clones":0,"num_object_copies":20178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10089,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8191,"num_write_kb":32764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9885","reported_seq":9963,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.224624+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:30.224624+0000","last_peered":"2024-09-11T01:18:30.224624+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:30.224624+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9885,"log_dups_size":0,"ondisk_log_size":9885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7946,"num_write_kb":31784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10019","reported_seq":10059,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.170025+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:30.170025+0000","last_peered":"2024-09-11T01:18:30.170025+0000","last_clean":"2024-09-11T01:18:30.170025+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:30.170025+0000","last_undegraded":"2024-09-11T01:18:30.170025+0000","last_fullsized":"2024-09-11T01:18:30.170025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41037824,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10019,"num_write_kb":40076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10003","reported_seq":10119,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.324262+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:31.324262+0000","last_peered":"2024-09-11T01:18:31.324262+0000","last_clean":"2024-09-11T01:18:31.324262+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:31.324262+0000","last_undegraded":"2024-09-11T01:18:31.324262+0000","last_fullsized":"2024-09-11T01:18:31.324262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40944099,"num_objects":9997,"num_object_clones":0,"num_object_copies":19994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6801,"num_write_kb":27204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9975","reported_seq":10104,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.210950+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:31.210950+0000","last_peered":"2024-09-11T01:18:31.210950+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:31.210950+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9975,"log_dups_size":0,"ondisk_log_size":9975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837603,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6463,"num_write_kb":25852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10271","reported_seq":10399,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.238381+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:31.238381+0000","last_peered":"2024-09-11T01:18:31.238381+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:31.238381+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42046406,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7024,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5984},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10070","reported_seq":10184,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.312744+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:31.312744+0000","last_peered":"2024-09-11T01:18:31.312744+0000","last_clean":"2024-09-11T01:18:31.312744+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:31.312744+0000","last_undegraded":"2024-09-11T01:18:31.312744+0000","last_fullsized":"2024-09-11T01:18:31.312744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7023,"num_write_kb":28092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10062","reported_seq":18400,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:33.971933+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:33.971933+0000","last_peered":"2024-09-11T01:18:33.971933+0000","last_clean":"2024-09-11T01:18:33.971933+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:33.971933+0000","last_undegraded":"2024-09-11T01:18:33.971933+0000","last_fullsized":"2024-09-11T01:18:33.971933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185763,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7929,"num_write_kb":31716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10171","reported_seq":10295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.267823+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:31.267823+0000","last_peered":"2024-09-11T01:18:31.267823+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:31.267823+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41627648,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6535,"num_write_kb":26140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6058},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9904","reported_seq":9993,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.173071+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:30.173071+0000","last_peered":"2024-09-11T01:18:30.173071+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:30.173071+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9904,"log_dups_size":0,"ondisk_log_size":9904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546787,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7673,"num_write_kb":30692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10152","reported_seq":10237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.189503+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:30.189503+0000","last_peered":"2024-09-11T01:18:30.189503+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:30.189503+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41558016,"num_objects":10146,"num_object_clones":0,"num_object_copies":20292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7789,"num_write_kb":31156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10061","reported_seq":11725,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:31.342704+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:31.342704+0000","last_peered":"2024-09-11T01:18:31.342704+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:31.342704+0000","last_undegraded":"2024-09-11T01:18:31.342704+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41201664,"num_objects":10059,"num_object_clones":0,"num_object_copies":20118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3395,"num_objects_unfound":0,"num_objects_dirty":10059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":3149824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6664},{"shards":"0,2","objects":3395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10189","reported_seq":10318,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.221985+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:31.221985+0000","last_peered":"2024-09-11T01:18:31.221985+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:31.221985+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701376,"num_objects":10181,"num_object_clones":0,"num_object_copies":20362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6582,"num_write_kb":26328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5989},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10174","reported_seq":10300,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.370254+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:31.370254+0000","last_peered":"2024-09-11T01:18:31.370254+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:31.370254+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41656320,"num_objects":10170,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6709,"num_write_kb":26836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6022},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9960","reported_seq":10088,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.310852+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:31.310852+0000","last_peered":"2024-09-11T01:18:31.310852+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:31.310852+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9960,"log_dups_size":0,"ondisk_log_size":9960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747974,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5796},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659104478,"num_objects":160923,"num_object_clones":0,"num_object_copies":321846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3395,"num_objects_unfound":0,"num_objects_dirty":160923,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":120058,"num_write_kb":480232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4895,"num_bytes_recovered":20042211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1341714432,"data_stored":1341562686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160395,"ondisk_log_size":160395,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701588,"kb_used_data":453316,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93670252,"statfs":{"total":96636764160,"available":95918338048,"internally_reserved":0,"allocated":464195584,"data_stored":463928279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552152,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93819688,"statfs":{"total":96636764160,"available":96071360512,"internally_reserved":0,"allocated":315432960,"data_stored":315118222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":873908,"kb_used_data":549348,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93497932,"statfs":{"total":96636764160,"available":95741882368,"internally_reserved":0,"allocated":562532352,"data_stored":562260970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,4,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562581504,"data_stored":562545374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315138048,"data_stored":315058562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463994880,"data_stored":463958750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:34.552 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:35.152 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:35.153 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:35.177 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-11T01:18:34.490841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659104497,"num_objects":160925,"num_object_clones":0,"num_object_copies":321850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3395,"num_objects_unfound":0,"num_objects_dirty":160925,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":120060,"num_write_kb":480234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4896,"num_bytes_recovered":20042230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160398,"ondisk_log_size":160398,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2127648,"kb_used_data":1310704,"kb_used_omap":12,"kb_used_meta":816883,"kb_avail":280987872,"statfs":{"total":289910292480,"available":287731580928,"internally_reserved":0,"allocated":1342160896,"data_stored":1341307471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":836488199},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,7,6,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18317312,"num_objects":4472,"num_object_clones":0,"num_object_copies":8944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-130,"num_objects_unfound":0,"num_objects_dirty":4472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4472,"num_write_kb":17888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002963"},"pg_stats":[{"pgid":"2.f","version":"41'10006","reported_seq":10074,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.260453+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:31.260453+0000","last_peered":"2024-09-11T01:18:31.260453+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:31.260453+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40984576,"num_objects":10006,"num_object_clones":0,"num_object_copies":20012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10006,"num_write_kb":40024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10093","reported_seq":10168,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.229901+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:30.229901+0000","last_peered":"2024-09-11T01:18:30.229901+0000","last_clean":"2024-09-11T01:18:30.229901+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:30.229901+0000","last_undegraded":"2024-09-11T01:18:30.229901+0000","last_fullsized":"2024-09-11T01:18:30.229901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41317318,"num_objects":10089,"num_object_clones":0,"num_object_copies":20178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10089,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8191,"num_write_kb":32764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9885","reported_seq":9963,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.224624+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:30.224624+0000","last_peered":"2024-09-11T01:18:30.224624+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:30.224624+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9885,"log_dups_size":0,"ondisk_log_size":9885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40472576,"num_objects":9881,"num_object_clones":0,"num_object_copies":19762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7946,"num_write_kb":31784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10019","reported_seq":10059,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.170025+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:30.170025+0000","last_peered":"2024-09-11T01:18:30.170025+0000","last_clean":"2024-09-11T01:18:30.170025+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:30.170025+0000","last_undegraded":"2024-09-11T01:18:30.170025+0000","last_fullsized":"2024-09-11T01:18:30.170025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41037824,"num_objects":10019,"num_object_clones":0,"num_object_copies":20038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10019,"num_write_kb":40076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10003","reported_seq":10119,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.324262+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:31.324262+0000","last_peered":"2024-09-11T01:18:31.324262+0000","last_clean":"2024-09-11T01:18:31.324262+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:31.324262+0000","last_undegraded":"2024-09-11T01:18:31.324262+0000","last_fullsized":"2024-09-11T01:18:31.324262+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40944099,"num_objects":9997,"num_object_clones":0,"num_object_copies":19994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6801,"num_write_kb":27204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'9975","reported_seq":10104,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.210950+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:31.210950+0000","last_peered":"2024-09-11T01:18:31.210950+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:31.210950+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":9975,"log_dups_size":0,"ondisk_log_size":9975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40837603,"num_objects":9971,"num_object_clones":0,"num_object_copies":19942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6463,"num_write_kb":25852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10271","reported_seq":10399,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.238381+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:31.238381+0000","last_peered":"2024-09-11T01:18:31.238381+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:31.238381+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42046406,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7024,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5984},{"shards":"0,2","objects":4283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10070","reported_seq":10184,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.312744+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:31.312744+0000","last_peered":"2024-09-11T01:18:31.312744+0000","last_clean":"2024-09-11T01:18:31.312744+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:31.312744+0000","last_undegraded":"2024-09-11T01:18:31.312744+0000","last_fullsized":"2024-09-11T01:18:31.312744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41238528,"num_objects":10068,"num_object_clones":0,"num_object_copies":20136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7023,"num_write_kb":28092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10062","reported_seq":18400,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:33.971933+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:33.971933+0000","last_peered":"2024-09-11T01:18:33.971933+0000","last_clean":"2024-09-11T01:18:33.971933+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:33.971933+0000","last_undegraded":"2024-09-11T01:18:33.971933+0000","last_fullsized":"2024-09-11T01:18:33.971933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185763,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7929,"num_write_kb":31716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893516+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893516+0000","last_peered":"2024-09-11T01:17:37.893516+0000","last_clean":"2024-09-11T01:17:37.893516+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893516+0000","last_undegraded":"2024-09-11T01:17:37.893516+0000","last_fullsized":"2024-09-11T01:17:37.893516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10171","reported_seq":10295,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.267823+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:31.267823+0000","last_peered":"2024-09-11T01:18:31.267823+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:31.267823+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41627648,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6535,"num_write_kb":26140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6058},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'9904","reported_seq":9993,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.173071+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:30.173071+0000","last_peered":"2024-09-11T01:18:30.173071+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:30.173071+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":9904,"log_dups_size":0,"ondisk_log_size":9904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40546787,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7673,"num_write_kb":30692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":5873}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10152","reported_seq":10237,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:30.189503+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:30.189503+0000","last_peered":"2024-09-11T01:18:30.189503+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:30.189503+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41558016,"num_objects":10146,"num_object_clones":0,"num_object_copies":20292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7789,"num_write_kb":31156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":5969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10061","reported_seq":11725,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:31.342704+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:31.342704+0000","last_peered":"2024-09-11T01:18:31.342704+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:31.342704+0000","last_undegraded":"2024-09-11T01:18:31.342704+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41201664,"num_objects":10059,"num_object_clones":0,"num_object_copies":20118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3395,"num_objects_unfound":0,"num_objects_dirty":10059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":769,"num_bytes_recovered":3149824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6664},{"shards":"0,2","objects":3395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10189","reported_seq":10318,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.221985+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:31.221985+0000","last_peered":"2024-09-11T01:18:31.221985+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:31.221985+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41701376,"num_objects":10181,"num_object_clones":0,"num_object_copies":20362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6582,"num_write_kb":26328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5989},{"shards":"0,2","objects":4192}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":55,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894424+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894424+0000","last_peered":"2024-09-11T01:17:37.894424+0000","last_clean":"2024-09-11T01:17:37.894424+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894424+0000","last_undegraded":"2024-09-11T01:17:37.894424+0000","last_fullsized":"2024-09-11T01:17:37.894424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10174","reported_seq":10300,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.370254+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:31.370254+0000","last_peered":"2024-09-11T01:18:31.370254+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:31.370254+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41656320,"num_objects":10170,"num_object_clones":0,"num_object_copies":20340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6709,"num_write_kb":26836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6022},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'9960","reported_seq":10088,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.310852+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:31.310852+0000","last_peered":"2024-09-11T01:18:31.310852+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:31.310852+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":9960,"log_dups_size":0,"ondisk_log_size":9960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747974,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5796},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659104478,"num_objects":160923,"num_object_clones":0,"num_object_copies":321846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3395,"num_objects_unfound":0,"num_objects_dirty":160923,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":120058,"num_write_kb":480232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4895,"num_bytes_recovered":20042211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1341714432,"data_stored":1341562686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160395,"ondisk_log_size":160395,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":701588,"kb_used_data":453316,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93670252,"statfs":{"total":96636764160,"available":95918338048,"internally_reserved":0,"allocated":464195584,"data_stored":463928279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552152,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93819688,"statfs":{"total":96636764160,"available":96071360512,"internally_reserved":0,"allocated":315432960,"data_stored":315118222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":873908,"kb_used_data":549348,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93497932,"statfs":{"total":96636764160,"available":95741882368,"internally_reserved":0,"allocated":562532352,"data_stored":562260970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,2,4,7,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562581504,"data_stored":562545374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315138048,"data_stored":315058562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":463994880,"data_stored":463958750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:35.178 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:35.178 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10006", 'reported_seq': 10074, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.260453+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:31.260453+0000', 'last_peered': '2024-09-11T01:18:31.260453+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:31.260453+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40984576, 'num_objects': 10006, 'num_object_clones': 0, 'num_object_copies': 20012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10006, 'num_write_kb': 40024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5883}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'9885", 'reported_seq': 9963, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:30.224624+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:30.224624+0000', 'last_peered': '2024-09-11T01:18:30.224624+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:30.224624+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 9885, 'log_dups_size': 0, 'ondisk_log_size': 9885, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40472576, 'num_objects': 9881, 'num_object_clones': 0, 'num_object_copies': 19762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4006, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7946, 'num_write_kb': 31784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4004}, {'shards': '2', 'objects': 5876}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'9975", 'reported_seq': 10104, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.210950+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:31.210950+0000', 'last_peered': '2024-09-11T01:18:31.210950+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:31.210950+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 9975, 'log_dups_size': 0, 'ondisk_log_size': 9975, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40837603, 'num_objects': 9971, 'num_object_clones': 0, 'num_object_copies': 19942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9971, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6463, 'num_write_kb': 25852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5856}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'10271", 'reported_seq': 10399, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.238381+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:31.238381+0000', 'last_peered': '2024-09-11T01:18:31.238381+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:31.238381+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'200", 'ondisk_log_start': "32'200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42046406, 'num_objects': 10267, 'num_object_clones': 0, 'num_object_copies': 20534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10267, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7024, 'num_write_kb': 28096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5984}, {'shards': '0,2', 'objects': 4283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10171", 'reported_seq': 10295, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.267823+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:31.267823+0000', 'last_peered': '2024-09-11T01:18:31.267823+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:31.267823+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41627648, 'num_objects': 10163, 'num_object_clones': 0, 'num_object_copies': 20326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6535, 'num_write_kb': 26140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6058}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:35.179 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'9904", 'reported_seq': 9993, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:30.173071+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:30.173071+0000', 'last_peered': '2024-09-11T01:18:30.173071+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:30.173071+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 9904, 'log_dups_size': 0, 'ondisk_log_size': 9904, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40546787, 'num_objects': 9900, 'num_object_clones': 0, 'num_object_copies': 19800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9900, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7673, 'num_write_kb': 30692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4027}, {'shards': '2', 'objects': 5873}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10152", 'reported_seq': 10237, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:30.189503+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:30.189503+0000', 'last_peered': '2024-09-11T01:18:30.189503+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:30.189503+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41558016, 'num_objects': 10146, 'num_object_clones': 0, 'num_object_copies': 20292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7789, 'num_write_kb': 31156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4177}, {'shards': '2', 'objects': 5969}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10061", 'reported_seq': 11725, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:31.342704+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:31.342704+0000', 'last_peered': '2024-09-11T01:18:31.342704+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:31.342704+0000', 'last_undegraded': '2024-09-11T01:18:31.342704+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41201664, 'num_objects': 10059, 'num_object_clones': 0, 'num_object_copies': 20118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3395, 'num_objects_unfound': 0, 'num_objects_dirty': 10059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6751, 'num_write_kb': 27004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 769, 'num_bytes_recovered': 3149824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6664}, {'shards': '0,2', 'objects': 3395}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'10189", 'reported_seq': 10318, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.221985+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:31.221985+0000', 'last_peered': '2024-09-11T01:18:31.221985+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:31.221985+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41701376, 'num_objects': 10181, 'num_object_clones': 0, 'num_object_copies': 20362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6582, 'num_write_kb': 26328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5989}, {'shards': '0,2', 'objects': 4192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10174", 'reported_seq': 10300, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.370254+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:31.370254+0000', 'last_peered': '2024-09-11T01:18:31.370254+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:31.370254+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41656320, 'num_objects': 10170, 'num_object_clones': 0, 'num_object_copies': 20340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6709, 'num_write_kb': 26836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6022}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:35.180 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'9960", 'reported_seq': 10088, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.310852+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:31.310852+0000', 'last_peered': '2024-09-11T01:18:31.310852+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:31.310852+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 9960, 'log_dups_size': 0, 'ondisk_log_size': 9960, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40747974, 'num_objects': 9950, 'num_object_clones': 0, 'num_object_copies': 19900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9950, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6617, 'num_write_kb': 26468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5796}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:35.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:35.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout: 239 20 162657 162637 2.65761 1.56641 0.0403738 0.0293886 2024-09-11T01:18:35.903 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:35.904 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41343/321850 objects degraded (12.845%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160925,"data_bytes":659104497,"bytes_used":2178711552,"bytes_avail":287731580928,"bytes_total":289910292480,"degraded_objects":41343,"degraded_total":321850,"degraded_ratio":0.12845424887369894,"misplaced_objects":3395,"misplaced_total":321850,"misplaced_ratio":0.010548392108124905,"recovering_objects_per_sec":10,"recovering_bytes_per_sec":44362,"recovering_keys_per_sec":0,"num_objects_recovered":130,"num_bytes_recovered":532480,"num_keys_recovered":0,"write_bytes_sec":1526065,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:35.937 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:35.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:36.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Total time run: 240.03 2024-09-11T01:18:36.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Total writes made: 163042 2024-09-11T01:18:36.361 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Write size: 4096 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Object size: 4096 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Bandwidth (MB/sec): 2.65335 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Stddev Bandwidth: 1.98113 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Max bandwidth (MB/sec): 12.1367 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Min bandwidth (MB/sec): 0 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Average IOPS: 679 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Stddev IOPS: 507.171 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Max IOPS: 3107 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Min IOPS: 0 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Average Latency(s): 0.0294383 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Stddev Latency(s): 0.0435231 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Max latency(s): 2.62159 2024-09-11T01:18:36.362 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Min latency(s): 0.00221854 2024-09-11T01:18:36.374 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T01:18:36.515 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:36.518 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:36.543 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-11T01:18:36.491332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666997489,"num_objects":162852,"num_object_clones":0,"num_object_copies":325704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162852,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":122011,"num_write_kb":487990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4987,"num_bytes_recovered":20414966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160549,"ondisk_log_size":160549,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2164664,"kb_used_data":1321352,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280950856,"statfs":{"total":289910292480,"available":287693676544,"internally_reserved":0,"allocated":1353064448,"data_stored":1352211023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":863489031},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,8,9,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25731072,"num_objects":6282,"num_object_clones":0,"num_object_copies":12564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-221,"num_objects_unfound":0,"num_objects_dirty":6282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6306,"num_write_kb":25176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":905216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002967"},"pg_stats":[{"pgid":"2.f","version":"41'10119","reported_seq":10187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.234051+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:36.234051+0000","last_peered":"2024-09-11T01:18:36.234051+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:36.234051+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5996},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10236","reported_seq":10311,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.095722+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:35.095722+0000","last_peered":"2024-09-11T01:18:35.095722+0000","last_clean":"2024-09-11T01:18:35.095722+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:35.095722+0000","last_undegraded":"2024-09-11T01:18:35.095722+0000","last_fullsized":"2024-09-11T01:18:35.095722+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41903046,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8334,"num_write_kb":33336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9999","reported_seq":10077,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:35.263236+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:35.263236+0000","last_peered":"2024-09-11T01:18:35.263236+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:35.263236+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9999,"log_dups_size":0,"ondisk_log_size":9999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40939520,"num_objects":9995,"num_object_clones":0,"num_object_copies":19990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8060,"num_write_kb":32240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5990}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10149","reported_seq":10189,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.156894+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:35.156894+0000","last_peered":"2024-09-11T01:18:35.156894+0000","last_clean":"2024-09-11T01:18:35.156894+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:35.156894+0000","last_undegraded":"2024-09-11T01:18:35.156894+0000","last_fullsized":"2024-09-11T01:18:35.156894+0000","mapping_epoch":31,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41570304,"num_objects":10149,"num_object_clones":0,"num_object_copies":20298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10149,"num_write_kb":40596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10148","reported_seq":10260,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.415993+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:36.415993+0000","last_peered":"2024-09-11T01:18:36.415993+0000","last_clean":"2024-09-11T01:18:36.415993+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:36.415993+0000","last_undegraded":"2024-09-11T01:18:36.415993+0000","last_fullsized":"2024-09-11T01:18:36.415993+0000","mapping_epoch":31,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488867,"num_objects":10130,"num_object_clones":0,"num_object_copies":20260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6946,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10106","reported_seq":10235,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.335091+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:36.335091+0000","last_peered":"2024-09-11T01:18:36.335091+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:36.335091+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41374179,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10410","reported_seq":10538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.315077+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:36.315077+0000","last_peered":"2024-09-11T01:18:36.315077+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:36.315077+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42607558,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7163,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6121},{"shards":"0,2","objects":4282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10215","reported_seq":10330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.394510+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:36.394510+0000","last_peered":"2024-09-11T01:18:36.394510+0000","last_clean":"2024-09-11T01:18:36.394510+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:36.394510+0000","last_undegraded":"2024-09-11T01:18:36.394510+0000","last_fullsized":"2024-09-11T01:18:36.394510+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7168,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10062","reported_seq":18400,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:33.971933+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:33.971933+0000","last_peered":"2024-09-11T01:18:33.971933+0000","last_clean":"2024-09-11T01:18:33.971933+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:33.971933+0000","last_undegraded":"2024-09-11T01:18:33.971933+0000","last_fullsized":"2024-09-11T01:18:33.971933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185763,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7929,"num_write_kb":31716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10301","reported_seq":10425,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.298339+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:36.298339+0000","last_peered":"2024-09-11T01:18:36.298339+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:36.298339+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42160128,"num_objects":10293,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10050","reported_seq":10138,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:35.251760+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:35.251760+0000","last_peered":"2024-09-11T01:18:35.251760+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:35.251760+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144803,"num_objects":10046,"num_object_clones":0,"num_object_copies":20092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7819,"num_write_kb":31276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":6019}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10267","reported_seq":10356,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:35.256902+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:35.256902+0000","last_peered":"2024-09-11T01:18:35.256902+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:35.256902+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7904,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":6084}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10177","reported_seq":12023,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:36.369217+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:36.369217+0000","last_peered":"2024-09-11T01:18:36.369217+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:36.369217+0000","last_undegraded":"2024-09-11T01:18:36.369217+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":3522560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6871},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10330","reported_seq":10459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.418724+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:36.418724+0000","last_peered":"2024-09-11T01:18:36.418724+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:36.418724+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6124},{"shards":"0,2","objects":4189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10302","reported_seq":10431,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.320602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:36.320602+0000","last_peered":"2024-09-11T01:18:36.320602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:36.320602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42180608,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6150},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10075","reported_seq":10205,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.336322+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:36.336322+0000","last_peered":"2024-09-11T01:18:36.336322+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:36.336322+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41219014,"num_objects":10065,"num_object_clones":0,"num_object_copies":20130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666997470,"num_objects":162850,"num_object_clones":0,"num_object_copies":325700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162850,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":122009,"num_write_kb":487988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4986,"num_bytes_recovered":20414947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1352359936,"data_stored":1352208190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160546,"ondisk_log_size":160546,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710024,"kb_used_data":457592,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93661816,"statfs":{"total":96636764160,"available":95909699584,"internally_reserved":0,"allocated":468574208,"data_stored":468306903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552152,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93819688,"statfs":{"total":96636764160,"available":96071360512,"internally_reserved":0,"allocated":315432960,"data_stored":315118222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902488,"kb_used_data":555720,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93469352,"statfs":{"total":96636764160,"available":95712616448,"internally_reserved":0,"allocated":569057280,"data_stored":568785898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315138048,"data_stored":315058562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":468393984,"data_stored":468357854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:39.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:40.126 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:40.127 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:40.147 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":194,"stamp":"2024-09-11T01:18:38.491836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666997489,"num_objects":162852,"num_object_clones":0,"num_object_copies":325704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162852,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":122011,"num_write_kb":487990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4987,"num_bytes_recovered":20414966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160549,"ondisk_log_size":160549,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2164664,"kb_used_data":1321352,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280950856,"statfs":{"total":289910292480,"available":287693676544,"internally_reserved":0,"allocated":1353064448,"data_stored":1352211023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":863489031},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,8,9,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17399808,"num_objects":4248,"num_object_clones":0,"num_object_copies":8496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-156,"num_objects_unfound":0,"num_objects_dirty":4248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4272,"num_write_kb":17040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":156,"num_bytes_recovered":638976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002989"},"pg_stats":[{"pgid":"2.f","version":"41'10119","reported_seq":10187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.234051+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:36.234051+0000","last_peered":"2024-09-11T01:18:36.234051+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:36.234051+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5996},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10236","reported_seq":10311,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.095722+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:35.095722+0000","last_peered":"2024-09-11T01:18:35.095722+0000","last_clean":"2024-09-11T01:18:35.095722+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:35.095722+0000","last_undegraded":"2024-09-11T01:18:35.095722+0000","last_fullsized":"2024-09-11T01:18:35.095722+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41903046,"num_objects":10232,"num_object_clones":0,"num_object_copies":20464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10232,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8334,"num_write_kb":33336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'9999","reported_seq":10077,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:35.263236+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:35.263236+0000","last_peered":"2024-09-11T01:18:35.263236+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:35.263236+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":9999,"log_dups_size":0,"ondisk_log_size":9999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40939520,"num_objects":9995,"num_object_clones":0,"num_object_copies":19990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4006,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8060,"num_write_kb":32240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4004},{"shards":"2","objects":5990}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10149","reported_seq":10189,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.156894+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:35.156894+0000","last_peered":"2024-09-11T01:18:35.156894+0000","last_clean":"2024-09-11T01:18:35.156894+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:35.156894+0000","last_undegraded":"2024-09-11T01:18:35.156894+0000","last_fullsized":"2024-09-11T01:18:35.156894+0000","mapping_epoch":31,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41570304,"num_objects":10149,"num_object_clones":0,"num_object_copies":20298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10149,"num_write_kb":40596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10148","reported_seq":10260,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.415993+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:36.415993+0000","last_peered":"2024-09-11T01:18:36.415993+0000","last_clean":"2024-09-11T01:18:36.415993+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:36.415993+0000","last_undegraded":"2024-09-11T01:18:36.415993+0000","last_fullsized":"2024-09-11T01:18:36.415993+0000","mapping_epoch":31,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488867,"num_objects":10130,"num_object_clones":0,"num_object_copies":20260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6946,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10106","reported_seq":10235,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.335091+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:36.335091+0000","last_peered":"2024-09-11T01:18:36.335091+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:36.335091+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41374179,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10410","reported_seq":10538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.315077+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:36.315077+0000","last_peered":"2024-09-11T01:18:36.315077+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:36.315077+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42607558,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7163,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6121},{"shards":"0,2","objects":4282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10215","reported_seq":10330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.394510+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:36.394510+0000","last_peered":"2024-09-11T01:18:36.394510+0000","last_clean":"2024-09-11T01:18:36.394510+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:36.394510+0000","last_undegraded":"2024-09-11T01:18:36.394510+0000","last_fullsized":"2024-09-11T01:18:36.394510+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7168,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10062","reported_seq":18400,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:33.971933+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:33.971933+0000","last_peered":"2024-09-11T01:18:33.971933+0000","last_clean":"2024-09-11T01:18:33.971933+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:33.971933+0000","last_undegraded":"2024-09-11T01:18:33.971933+0000","last_fullsized":"2024-09-11T01:18:33.971933+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185763,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7929,"num_write_kb":31716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10301","reported_seq":10425,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.298339+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:36.298339+0000","last_peered":"2024-09-11T01:18:36.298339+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:36.298339+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42160128,"num_objects":10293,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10050","reported_seq":10138,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:35.251760+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:35.251760+0000","last_peered":"2024-09-11T01:18:35.251760+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:35.251760+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144803,"num_objects":10046,"num_object_clones":0,"num_object_copies":20092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7819,"num_write_kb":31276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4027},{"shards":"2","objects":6019}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10267","reported_seq":10356,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:35.256902+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:35.256902+0000","last_peered":"2024-09-11T01:18:35.256902+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:35.256902+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42029056,"num_objects":10261,"num_object_clones":0,"num_object_copies":20522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7904,"num_write_kb":31616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4177},{"shards":"2","objects":6084}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10177","reported_seq":12023,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:36.369217+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:36.369217+0000","last_peered":"2024-09-11T01:18:36.369217+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:36.369217+0000","last_undegraded":"2024-09-11T01:18:36.369217+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":3522560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6871},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10330","reported_seq":10459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.418724+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:36.418724+0000","last_peered":"2024-09-11T01:18:36.418724+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:36.418724+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6124},{"shards":"0,2","objects":4189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10302","reported_seq":10431,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.320602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:36.320602+0000","last_peered":"2024-09-11T01:18:36.320602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:36.320602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42180608,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6150},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10075","reported_seq":10205,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.336322+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:36.336322+0000","last_peered":"2024-09-11T01:18:36.336322+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:36.336322+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41219014,"num_objects":10065,"num_object_clones":0,"num_object_copies":20130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666997470,"num_objects":162850,"num_object_clones":0,"num_object_copies":325700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41343,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162850,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":122009,"num_write_kb":487988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4986,"num_bytes_recovered":20414947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1352359936,"data_stored":1352208190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160546,"ondisk_log_size":160546,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710024,"kb_used_data":457592,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93661816,"statfs":{"total":96636764160,"available":95909699584,"internally_reserved":0,"allocated":468574208,"data_stored":468306903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":258469550},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789992,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552152,"kb_used_data":308040,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93819688,"statfs":{"total":96636764160,"available":96071360512,"internally_reserved":0,"allocated":315432960,"data_stored":315118222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902488,"kb_used_data":555720,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93469352,"statfs":{"total":96636764160,"available":95712616448,"internally_reserved":0,"allocated":569057280,"data_stored":568785898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315138048,"data_stored":315058562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":468393984,"data_stored":468357854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:40.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:40.730 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:40.733 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:40.752 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-11T01:18:40.492332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665240329,"num_objects":162424,"num_object_clones":0,"num_object_copies":324848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41339,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162424,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":122821,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4991,"num_bytes_recovered":20414966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2173400,"kb_used_data":1321768,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280942120,"statfs":{"total":289910292480,"available":287684730880,"internally_reserved":0,"allocated":1353490432,"data_stored":1352628863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":872008711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,0,6,12,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15642648,"num_objects":3820,"num_object_clones":0,"num_object_copies":7640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4,"num_objects_misplaced":-156,"num_objects_unfound":0,"num_objects_dirty":3820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5082,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":638976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002971"},"pg_stats":[{"pgid":"2.f","version":"41'10119","reported_seq":10187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.234051+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:36.234051+0000","last_peered":"2024-09-11T01:18:36.234051+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:36.234051+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5996},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10379","reported_seq":10454,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.182114+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:40.182114+0000","last_peered":"2024-09-11T01:18:40.182114+0000","last_clean":"2024-09-11T01:18:40.182114+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:40.182114+0000","last_undegraded":"2024-09-11T01:18:40.182114+0000","last_fullsized":"2024-09-11T01:18:40.182114+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8477,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10136","reported_seq":10223,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.281692+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:40.281692+0000","last_peered":"2024-09-11T01:18:40.281692+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:40.281692+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40587288,"num_objects":9910,"num_object_clones":0,"num_object_copies":19820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8197,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3893},{"shards":"2","objects":5909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10293","reported_seq":10333,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.197894+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:40.197894+0000","last_peered":"2024-09-11T01:18:40.197894+0000","last_clean":"2024-09-11T01:18:40.197894+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:40.197894+0000","last_undegraded":"2024-09-11T01:18:40.197894+0000","last_fullsized":"2024-09-11T01:18:40.197894+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41242624,"num_objects":10069,"num_object_clones":0,"num_object_copies":20138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10293,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10148","reported_seq":10260,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.415993+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:36.415993+0000","last_peered":"2024-09-11T01:18:36.415993+0000","last_clean":"2024-09-11T01:18:36.415993+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:36.415993+0000","last_undegraded":"2024-09-11T01:18:36.415993+0000","last_fullsized":"2024-09-11T01:18:36.415993+0000","mapping_epoch":31,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488867,"num_objects":10130,"num_object_clones":0,"num_object_copies":20260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6946,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10106","reported_seq":10235,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.335091+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:36.335091+0000","last_peered":"2024-09-11T01:18:36.335091+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:36.335091+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41374179,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10410","reported_seq":10538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.315077+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:36.315077+0000","last_peered":"2024-09-11T01:18:36.315077+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:36.315077+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42607558,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7163,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6121},{"shards":"0,2","objects":4282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10215","reported_seq":10330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.394510+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:36.394510+0000","last_peered":"2024-09-11T01:18:36.394510+0000","last_clean":"2024-09-11T01:18:36.394510+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:36.394510+0000","last_undegraded":"2024-09-11T01:18:36.394510+0000","last_fullsized":"2024-09-11T01:18:36.394510+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7168,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10194","reported_seq":18532,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:39.001608+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:39.001608+0000","last_peered":"2024-09-11T01:18:39.001608+0000","last_clean":"2024-09-11T01:18:39.001608+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:39.001608+0000","last_undegraded":"2024-09-11T01:18:39.001608+0000","last_fullsized":"2024-09-11T01:18:39.001608+0000","mapping_epoch":40,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185763,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8061,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10301","reported_seq":10425,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.298339+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:36.298339+0000","last_peered":"2024-09-11T01:18:36.298339+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:36.298339+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42160128,"num_objects":10293,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10181","reported_seq":10271,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.299314+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:40.299314+0000","last_peered":"2024-09-11T01:18:40.299314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:40.299314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7950,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3918},{"shards":"2","objects":5932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10390","reported_seq":10479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.278889+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:40.278889+0000","last_peered":"2024-09-11T01:18:40.278889+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:40.278889+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41664512,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8027,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4071},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10177","reported_seq":12023,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:36.369217+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:36.369217+0000","last_peered":"2024-09-11T01:18:36.369217+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:36.369217+0000","last_undegraded":"2024-09-11T01:18:36.369217+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":3522560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6871},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10330","reported_seq":10459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.418724+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:36.418724+0000","last_peered":"2024-09-11T01:18:36.418724+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:36.418724+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6124},{"shards":"0,2","objects":4189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10302","reported_seq":10431,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.320602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:36.320602+0000","last_peered":"2024-09-11T01:18:36.320602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:36.320602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42180608,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6150},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10075","reported_seq":10205,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.336322+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:36.336322+0000","last_peered":"2024-09-11T01:18:36.336322+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:36.336322+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41219014,"num_objects":10065,"num_object_clones":0,"num_object_copies":20130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":665240310,"num_objects":162422,"num_object_clones":0,"num_object_copies":324844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41339,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162422,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":122819,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4990,"num_bytes_recovered":20414947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1352130560,"data_stored":1351970670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712004,"kb_used_data":455412,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93659836,"statfs":{"total":96636764160,"available":95907672064,"internally_reserved":0,"allocated":466341888,"data_stored":466070511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789993,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558908,"kb_used_data":310636,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93812932,"statfs":{"total":96636764160,"available":96064442368,"internally_reserved":0,"allocated":318091264,"data_stored":317772454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902488,"kb_used_data":555720,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93469352,"statfs":{"total":96636764160,"available":95712616448,"internally_reserved":0,"allocated":569057280,"data_stored":568785898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317612032,"data_stored":317528474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465690624,"data_stored":465650422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10119", 'reported_seq': 10187, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.234051+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:36.234051+0000', 'last_peered': '2024-09-11T01:18:36.234051+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:36.234051+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41447424, 'num_objects': 10119, 'num_object_clones': 0, 'num_object_copies': 20238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10119, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5996}, {'shards': '0,2', 'objects': 4123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10136", 'reported_seq': 10223, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:40.281692+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:40.281692+0000', 'last_peered': '2024-09-11T01:18:40.281692+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:40.281692+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40587288, 'num_objects': 9910, 'num_object_clones': 0, 'num_object_copies': 19820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9910, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8197, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3893}, {'shards': '2', 'objects': 5909}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'10106", 'reported_seq': 10235, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.335091+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:36.335091+0000', 'last_peered': '2024-09-11T01:18:36.335091+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:36.335091+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41374179, 'num_objects': 10102, 'num_object_clones': 0, 'num_object_copies': 20204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4116, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10102, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6594, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5987}, {'shards': '0,2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'10410", 'reported_seq': 10538, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.315077+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:36.315077+0000', 'last_peered': '2024-09-11T01:18:36.315077+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:36.315077+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42607558, 'num_objects': 10404, 'num_object_clones': 0, 'num_object_copies': 20808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4283, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10404, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7163, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6121}, {'shards': '0,2', 'objects': 4282}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:40.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10301", 'reported_seq': 10425, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.298339+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:36.298339+0000', 'last_peered': '2024-09-11T01:18:36.298339+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:36.298339+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42160128, 'num_objects': 10293, 'num_object_clones': 0, 'num_object_copies': 20586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6665, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6188}, {'shards': '0,2', 'objects': 4105}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10181", 'reported_seq': 10271, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:40.299314+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:40.299314+0000', 'last_peered': '2024-09-11T01:18:40.299314+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:40.299314+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40788451, 'num_objects': 9959, 'num_object_clones': 0, 'num_object_copies': 19918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9959, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7950, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3918}, {'shards': '2', 'objects': 5932}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10390", 'reported_seq': 10479, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:40.278889+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:40.278889+0000', 'last_peered': '2024-09-11T01:18:40.278889+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:40.278889+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41664512, 'num_objects': 10172, 'num_object_clones': 0, 'num_object_copies': 20344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8027, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4071}, {'shards': '2', 'objects': 5995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10177", 'reported_seq': 12023, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:36.369217+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:36.369217+0000', 'last_peered': '2024-09-11T01:18:36.369217+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:36.369217+0000', 'last_undegraded': '2024-09-11T01:18:36.369217+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41676800, 'num_objects': 10175, 'num_object_clones': 0, 'num_object_copies': 20350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3304, 'num_objects_unfound': 0, 'num_objects_dirty': 10175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6867, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 860, 'num_bytes_recovered': 3522560, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6871}, {'shards': '0,2', 'objects': 3304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'10330", 'reported_seq': 10459, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.418724+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:36.418724+0000', 'last_peered': '2024-09-11T01:18:36.418724+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:36.418724+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42254336, 'num_objects': 10316, 'num_object_clones': 0, 'num_object_copies': 20632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6723, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6124}, {'shards': '0,2', 'objects': 4189}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:40.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10302", 'reported_seq': 10431, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.320602+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:36.320602+0000', 'last_peered': '2024-09-11T01:18:36.320602+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:36.320602+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42180608, 'num_objects': 10298, 'num_object_clones': 0, 'num_object_copies': 20596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6837, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6150}, {'shards': '0,2', 'objects': 4148}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:40.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10075", 'reported_seq': 10205, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:36.336322+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:36.336322+0000', 'last_peered': '2024-09-11T01:18:36.336322+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:36.336322+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41219014, 'num_objects': 10065, 'num_object_clones': 0, 'num_object_copies': 20130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4156, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10065, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6732, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5911}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:40.756 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:41.486 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:41.487 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41339/324848 objects degraded (12.726%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":321,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":162424,"data_bytes":665240329,"bytes_used":2225561600,"bytes_avail":287684730880,"bytes_total":289910292480,"degraded_objects":41339,"degraded_total":324848,"degraded_ratio":0.12725643993498498,"misplaced_objects":3304,"misplaced_total":324848,"misplaced_ratio":0.010170910702851796,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":53234,"recovering_keys_per_sec":0,"num_objects_recovered":160,"num_bytes_recovered":638976,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":1518642,"read_op_per_sec":0,"write_op_per_sec":423},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:41.512 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:41.513 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:42.089 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:42.092 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:42.114 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-11T01:18:40.492332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665240329,"num_objects":162424,"num_object_clones":0,"num_object_copies":324848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41339,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162424,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":122821,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4991,"num_bytes_recovered":20414966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160759,"ondisk_log_size":160759,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2173400,"kb_used_data":1321768,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280942120,"statfs":{"total":289910292480,"available":287684730880,"internally_reserved":0,"allocated":1353490432,"data_stored":1352628863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":872008711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,0,6,12,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15642648,"num_objects":3820,"num_object_clones":0,"num_object_copies":7640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4,"num_objects_misplaced":-156,"num_objects_unfound":0,"num_objects_dirty":3820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5082,"num_write_kb":17801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":160,"num_bytes_recovered":638976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002971"},"pg_stats":[{"pgid":"2.f","version":"41'10119","reported_seq":10187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.234051+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:36.234051+0000","last_peered":"2024-09-11T01:18:36.234051+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:36.234051+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41447424,"num_objects":10119,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10119,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5996},{"shards":"0,2","objects":4123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10379","reported_seq":10454,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.182114+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:40.182114+0000","last_peered":"2024-09-11T01:18:40.182114+0000","last_clean":"2024-09-11T01:18:40.182114+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:40.182114+0000","last_undegraded":"2024-09-11T01:18:40.182114+0000","last_fullsized":"2024-09-11T01:18:40.182114+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8477,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10136","reported_seq":10223,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.281692+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:40.281692+0000","last_peered":"2024-09-11T01:18:40.281692+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:40.281692+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40587288,"num_objects":9910,"num_object_clones":0,"num_object_copies":19820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8197,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3893},{"shards":"2","objects":5909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10293","reported_seq":10333,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.197894+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:40.197894+0000","last_peered":"2024-09-11T01:18:40.197894+0000","last_clean":"2024-09-11T01:18:40.197894+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:40.197894+0000","last_undegraded":"2024-09-11T01:18:40.197894+0000","last_fullsized":"2024-09-11T01:18:40.197894+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41242624,"num_objects":10069,"num_object_clones":0,"num_object_copies":20138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10293,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10148","reported_seq":10260,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.415993+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:36.415993+0000","last_peered":"2024-09-11T01:18:36.415993+0000","last_clean":"2024-09-11T01:18:36.415993+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:36.415993+0000","last_undegraded":"2024-09-11T01:18:36.415993+0000","last_fullsized":"2024-09-11T01:18:36.415993+0000","mapping_epoch":31,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41488867,"num_objects":10130,"num_object_clones":0,"num_object_copies":20260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10130,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6946,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10106","reported_seq":10235,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.335091+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:36.335091+0000","last_peered":"2024-09-11T01:18:36.335091+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:36.335091+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41374179,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4116,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5987},{"shards":"0,2","objects":4114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10410","reported_seq":10538,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.315077+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:36.315077+0000","last_peered":"2024-09-11T01:18:36.315077+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:36.315077+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42607558,"num_objects":10404,"num_object_clones":0,"num_object_copies":20808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10404,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7163,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6121},{"shards":"0,2","objects":4282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10215","reported_seq":10330,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.394510+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:36.394510+0000","last_peered":"2024-09-11T01:18:36.394510+0000","last_clean":"2024-09-11T01:18:36.394510+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:36.394510+0000","last_undegraded":"2024-09-11T01:18:36.394510+0000","last_fullsized":"2024-09-11T01:18:36.394510+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41816064,"num_objects":10209,"num_object_clones":0,"num_object_copies":20418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7168,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10194","reported_seq":18532,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:39.001608+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:39.001608+0000","last_peered":"2024-09-11T01:18:39.001608+0000","last_clean":"2024-09-11T01:18:39.001608+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:39.001608+0000","last_undegraded":"2024-09-11T01:18:39.001608+0000","last_fullsized":"2024-09-11T01:18:39.001608+0000","mapping_epoch":40,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185763,"num_objects":10056,"num_object_clones":0,"num_object_copies":20112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10056,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8061,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10301","reported_seq":10425,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.298339+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:36.298339+0000","last_peered":"2024-09-11T01:18:36.298339+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:36.298339+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42160128,"num_objects":10293,"num_object_clones":0,"num_object_copies":20586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6665,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6188},{"shards":"0,2","objects":4105}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10181","reported_seq":10271,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.299314+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:40.299314+0000","last_peered":"2024-09-11T01:18:40.299314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:40.299314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7950,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3918},{"shards":"2","objects":5932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10390","reported_seq":10479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.278889+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:40.278889+0000","last_peered":"2024-09-11T01:18:40.278889+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:40.278889+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41664512,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8027,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4071},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10177","reported_seq":12023,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:36.369217+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:36.369217+0000","last_peered":"2024-09-11T01:18:36.369217+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:36.369217+0000","last_undegraded":"2024-09-11T01:18:36.369217+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41676800,"num_objects":10175,"num_object_clones":0,"num_object_copies":20350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":10175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":3522560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6871},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10330","reported_seq":10459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.418724+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:36.418724+0000","last_peered":"2024-09-11T01:18:36.418724+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:36.418724+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42254336,"num_objects":10316,"num_object_clones":0,"num_object_copies":20632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6723,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6124},{"shards":"0,2","objects":4189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10302","reported_seq":10431,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.320602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:36.320602+0000","last_peered":"2024-09-11T01:18:36.320602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:36.320602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42180608,"num_objects":10298,"num_object_clones":0,"num_object_copies":20596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6837,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6150},{"shards":"0,2","objects":4148}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10075","reported_seq":10205,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.336322+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:36.336322+0000","last_peered":"2024-09-11T01:18:36.336322+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:36.336322+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41219014,"num_objects":10065,"num_object_clones":0,"num_object_copies":20130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4156,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6732,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":665240310,"num_objects":162422,"num_object_clones":0,"num_object_copies":324844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41339,"num_objects_misplaced":3304,"num_objects_unfound":0,"num_objects_dirty":162422,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":122819,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4990,"num_bytes_recovered":20414947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1352130560,"data_stored":1351970670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712004,"kb_used_data":455412,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93659836,"statfs":{"total":96636764160,"available":95907672064,"internally_reserved":0,"allocated":466341888,"data_stored":466070511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789993,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558908,"kb_used_data":310636,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93812932,"statfs":{"total":96636764160,"available":96064442368,"internally_reserved":0,"allocated":318091264,"data_stored":317772454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":902488,"kb_used_data":555720,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93469352,"statfs":{"total":96636764160,"available":95712616448,"internally_reserved":0,"allocated":569057280,"data_stored":568785898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317612032,"data_stored":317528474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465690624,"data_stored":465650422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:45.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:45.669 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:45.669 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:45.698 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-11T01:18:44.493324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659145481,"num_objects":160936,"num_object_clones":0,"num_object_copies":321872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41318,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":160936,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":124309,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5096,"num_bytes_recovered":20759030,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2158104,"kb_used_data":1314784,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280957416,"statfs":{"total":289910292480,"available":287700393984,"internally_reserved":0,"allocated":1346338816,"data_stored":1345477247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":863489031},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,4,6,9,2,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":589848,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4383,"num_write_kb":9053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"2.f","version":"41'10234","reported_seq":10310,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.427992+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:41.427992+0000","last_peered":"2024-09-11T01:18:41.427992+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:41.427992+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976384,"num_objects":10004,"num_object_clones":0,"num_object_copies":20008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10234,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5885},{"shards":"0,2","objects":4008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10379","reported_seq":10454,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.182114+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:40.182114+0000","last_peered":"2024-09-11T01:18:40.182114+0000","last_clean":"2024-09-11T01:18:40.182114+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:40.182114+0000","last_undegraded":"2024-09-11T01:18:40.182114+0000","last_fullsized":"2024-09-11T01:18:40.182114+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8477,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10136","reported_seq":10223,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.281692+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:40.281692+0000","last_peered":"2024-09-11T01:18:40.281692+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:40.281692+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40587288,"num_objects":9910,"num_object_clones":0,"num_object_copies":19820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8197,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3893},{"shards":"2","objects":5909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10293","reported_seq":10333,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.197894+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:40.197894+0000","last_peered":"2024-09-11T01:18:40.197894+0000","last_clean":"2024-09-11T01:18:40.197894+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:40.197894+0000","last_undegraded":"2024-09-11T01:18:40.197894+0000","last_fullsized":"2024-09-11T01:18:40.197894+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41242624,"num_objects":10069,"num_object_clones":0,"num_object_copies":20138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10293,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10273","reported_seq":10390,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.261360+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:41.261360+0000","last_peered":"2024-09-11T01:18:41.261360+0000","last_clean":"2024-09-11T01:18:41.261360+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:41.261360+0000","last_undegraded":"2024-09-11T01:18:41.261360+0000","last_fullsized":"2024-09-11T01:18:41.261360+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976867,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7071,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10249","reported_seq":10383,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.426627+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:41.426627+0000","last_peered":"2024-09-11T01:18:41.426627+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:41.426627+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6737,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10538","reported_seq":10671,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.378508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:41.378508+0000","last_peered":"2024-09-11T01:18:41.378508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:41.378508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42083270,"num_objects":10276,"num_object_clones":0,"num_object_copies":20552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7291,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5995},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10338","reported_seq":10453,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.403561+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:41.403561+0000","last_peered":"2024-09-11T01:18:41.403561+0000","last_clean":"2024-09-11T01:18:41.403561+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:41.403561+0000","last_undegraded":"2024-09-11T01:18:41.403561+0000","last_fullsized":"2024-09-11T01:18:41.403561+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41312256,"num_objects":10086,"num_object_clones":0,"num_object_copies":20172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7291,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10337","reported_seq":18675,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.041014+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:44.041014+0000","last_peered":"2024-09-11T01:18:44.041014+0000","last_clean":"2024-09-11T01:18:44.041014+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:44.041014+0000","last_undegraded":"2024-09-11T01:18:44.041014+0000","last_fullsized":"2024-09-11T01:18:44.041014+0000","mapping_epoch":40,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40600035,"num_objects":9913,"num_object_clones":0,"num_object_copies":19826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8204,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10440","reported_seq":10572,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.388641+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:41.388641+0000","last_peered":"2024-09-11T01:18:41.388641+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:41.388641+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6804,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6053},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10181","reported_seq":10271,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.299314+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:40.299314+0000","last_peered":"2024-09-11T01:18:40.299314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:40.299314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7950,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3918},{"shards":"2","objects":5932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10390","reported_seq":10479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.278889+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:40.278889+0000","last_peered":"2024-09-11T01:18:40.278889+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:40.278889+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41664512,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8027,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4071},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10326","reported_seq":12340,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:41.457449+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:41.457449+0000","last_peered":"2024-09-11T01:18:41.457449+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:41.457449+0000","last_undegraded":"2024-09-11T01:18:41.457449+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6806},{"shards":"0,2","objects":3220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10475","reported_seq":10612,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.428878+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:41.428878+0000","last_peered":"2024-09-11T01:18:41.428878+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:41.428878+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660416,"num_objects":10171,"num_object_clones":0,"num_object_copies":20342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6868,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5984},{"shards":"0,2","objects":4044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10437","reported_seq":10572,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.374977+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:41.374977+0000","last_peered":"2024-09-11T01:18:41.374977+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:41.374977+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41627648,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6972,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":4013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10218","reported_seq":10347,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.418802+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:41.418802+0000","last_peered":"2024-09-11T01:18:41.418802+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:41.418802+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40633286,"num_objects":9922,"num_object_clones":0,"num_object_copies":19844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6875,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5769},{"shards":"0,2","objects":4011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659145462,"num_objects":160934,"num_object_clones":0,"num_object_copies":321868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41318,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":160934,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":124307,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5095,"num_bytes_recovered":20759011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1344679936,"data_stored":1344520046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712004,"kb_used_data":455412,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93659836,"statfs":{"total":96636764160,"available":95907672064,"internally_reserved":0,"allocated":466341888,"data_stored":466070511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789994,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562296,"kb_used_data":309864,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93809544,"statfs":{"total":96636764160,"available":96060973056,"internally_reserved":0,"allocated":317300736,"data_stored":316981926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":883804,"kb_used_data":549508,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93488036,"statfs":{"total":96636764160,"available":95731748864,"internally_reserved":0,"allocated":562696192,"data_stored":562424810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,3,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562089984,"data_stored":562053854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316899328,"data_stored":316815770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465690624,"data_stored":465650422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:45.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:46.258 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:46.259 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:46.280 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-11T01:18:44.493324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659145481,"num_objects":160936,"num_object_clones":0,"num_object_copies":321872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41318,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":160936,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":124309,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5096,"num_bytes_recovered":20759030,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2158104,"kb_used_data":1314784,"kb_used_omap":12,"kb_used_meta":843251,"kb_avail":280957416,"statfs":{"total":289910292480,"available":287700393984,"internally_reserved":0,"allocated":1346338816,"data_stored":1345477247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":863489031},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,4,6,9,2,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":589848,"num_objects":145,"num_object_clones":0,"num_object_copies":290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4383,"num_write_kb":9053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003041"},"pg_stats":[{"pgid":"2.f","version":"41'10234","reported_seq":10310,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.427992+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:41.427992+0000","last_peered":"2024-09-11T01:18:41.427992+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:41.427992+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976384,"num_objects":10004,"num_object_clones":0,"num_object_copies":20008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10234,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5885},{"shards":"0,2","objects":4008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10379","reported_seq":10454,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.182114+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:40.182114+0000","last_peered":"2024-09-11T01:18:40.182114+0000","last_clean":"2024-09-11T01:18:40.182114+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:40.182114+0000","last_undegraded":"2024-09-11T01:18:40.182114+0000","last_fullsized":"2024-09-11T01:18:40.182114+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8477,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10136","reported_seq":10223,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.281692+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:40.281692+0000","last_peered":"2024-09-11T01:18:40.281692+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:40.281692+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40587288,"num_objects":9910,"num_object_clones":0,"num_object_copies":19820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8197,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3893},{"shards":"2","objects":5909}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10293","reported_seq":10333,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.197894+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:40.197894+0000","last_peered":"2024-09-11T01:18:40.197894+0000","last_clean":"2024-09-11T01:18:40.197894+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:40.197894+0000","last_undegraded":"2024-09-11T01:18:40.197894+0000","last_fullsized":"2024-09-11T01:18:40.197894+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41242624,"num_objects":10069,"num_object_clones":0,"num_object_copies":20138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10293,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10273","reported_seq":10390,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.261360+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:41.261360+0000","last_peered":"2024-09-11T01:18:41.261360+0000","last_clean":"2024-09-11T01:18:41.261360+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:41.261360+0000","last_undegraded":"2024-09-11T01:18:41.261360+0000","last_fullsized":"2024-09-11T01:18:41.261360+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976867,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7071,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10249","reported_seq":10383,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.426627+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:41.426627+0000","last_peered":"2024-09-11T01:18:41.426627+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:41.426627+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6737,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10538","reported_seq":10671,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.378508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:41.378508+0000","last_peered":"2024-09-11T01:18:41.378508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:41.378508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42083270,"num_objects":10276,"num_object_clones":0,"num_object_copies":20552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7291,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5995},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10338","reported_seq":10453,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.403561+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:41.403561+0000","last_peered":"2024-09-11T01:18:41.403561+0000","last_clean":"2024-09-11T01:18:41.403561+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:41.403561+0000","last_undegraded":"2024-09-11T01:18:41.403561+0000","last_fullsized":"2024-09-11T01:18:41.403561+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41312256,"num_objects":10086,"num_object_clones":0,"num_object_copies":20172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7291,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10337","reported_seq":18675,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.041014+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:44.041014+0000","last_peered":"2024-09-11T01:18:44.041014+0000","last_clean":"2024-09-11T01:18:44.041014+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:44.041014+0000","last_undegraded":"2024-09-11T01:18:44.041014+0000","last_fullsized":"2024-09-11T01:18:44.041014+0000","mapping_epoch":40,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40600035,"num_objects":9913,"num_object_clones":0,"num_object_copies":19826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8204,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10440","reported_seq":10572,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.388641+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:41.388641+0000","last_peered":"2024-09-11T01:18:41.388641+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:41.388641+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6804,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6053},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10181","reported_seq":10271,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.299314+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:40.299314+0000","last_peered":"2024-09-11T01:18:40.299314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:40.299314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'100","ondisk_log_start":"32'100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4028,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7950,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3918},{"shards":"2","objects":5932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10390","reported_seq":10479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:40.278889+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:40.278889+0000","last_peered":"2024-09-11T01:18:40.278889+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:40.278889+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41664512,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8027,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4071},{"shards":"2","objects":5995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10326","reported_seq":12340,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:41.457449+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:41.457449+0000","last_peered":"2024-09-11T01:18:41.457449+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:41.457449+0000","last_undegraded":"2024-09-11T01:18:41.457449+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6806},{"shards":"0,2","objects":3220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10475","reported_seq":10612,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.428878+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:41.428878+0000","last_peered":"2024-09-11T01:18:41.428878+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:41.428878+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660416,"num_objects":10171,"num_object_clones":0,"num_object_copies":20342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6868,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5984},{"shards":"0,2","objects":4044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10437","reported_seq":10572,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.374977+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:41.374977+0000","last_peered":"2024-09-11T01:18:41.374977+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:41.374977+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41627648,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6972,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":4013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10218","reported_seq":10347,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.418802+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:41.418802+0000","last_peered":"2024-09-11T01:18:41.418802+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:41.418802+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40633286,"num_objects":9922,"num_object_clones":0,"num_object_copies":19844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6875,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5769},{"shards":"0,2","objects":4011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":659145462,"num_objects":160934,"num_object_clones":0,"num_object_copies":321868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41318,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":160934,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":124307,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5095,"num_bytes_recovered":20759011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1344679936,"data_stored":1344520046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712004,"kb_used_data":455412,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93659836,"statfs":{"total":96636764160,"available":95907672064,"internally_reserved":0,"allocated":466341888,"data_stored":466070511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":262729390},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789994,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562296,"kb_used_data":309864,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93809544,"statfs":{"total":96636764160,"available":96060973056,"internally_reserved":0,"allocated":317300736,"data_stored":316981926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":883804,"kb_used_data":549508,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93488036,"statfs":{"total":96636764160,"available":95731748864,"internally_reserved":0,"allocated":562696192,"data_stored":562424810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,3,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562089984,"data_stored":562053854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316899328,"data_stored":316815770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465690624,"data_stored":465650422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:46.281 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:46.281 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10234", 'reported_seq': 10310, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.427992+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:41.427992+0000', 'last_peered': '2024-09-11T01:18:41.427992+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:41.427992+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'200", 'ondisk_log_start': "32'200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40976384, 'num_objects': 10004, 'num_object_clones': 0, 'num_object_copies': 20008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4119, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10234, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5885}, {'shards': '0,2', 'objects': 4008}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.281 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:46.281 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10136", 'reported_seq': 10223, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:40.281692+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:40.281692+0000', 'last_peered': '2024-09-11T01:18:40.281692+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:40.281692+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40587288, 'num_objects': 9910, 'num_object_clones': 0, 'num_object_copies': 19820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9910, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8197, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3893}, {'shards': '2', 'objects': 5909}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'10249", 'reported_seq': 10383, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.426627+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:41.426627+0000', 'last_peered': '2024-09-11T01:18:41.426627+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:41.426627+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'200", 'ondisk_log_start': "32'200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40788451, 'num_objects': 9959, 'num_object_clones': 0, 'num_object_copies': 19918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9959, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6737, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5846}, {'shards': '0,2', 'objects': 3971}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'10538", 'reported_seq': 10671, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.378508+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:41.378508+0000', 'last_peered': '2024-09-11T01:18:41.378508+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:41.378508+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42083270, 'num_objects': 10276, 'num_object_clones': 0, 'num_object_copies': 20552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4281, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10276, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7291, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5995}, {'shards': '0,2', 'objects': 4154}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10440", 'reported_seq': 10572, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.388641+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:41.388641+0000', 'last_peered': '2024-09-11T01:18:41.388641+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:41.388641+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41590784, 'num_objects': 10154, 'num_object_clones': 0, 'num_object_copies': 20308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6804, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6053}, {'shards': '0,2', 'objects': 3966}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10181", 'reported_seq': 10271, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:40.299314+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:40.299314+0000', 'last_peered': '2024-09-11T01:18:40.299314+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:40.299314+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'100", 'ondisk_log_start': "32'100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40788451, 'num_objects': 9959, 'num_object_clones': 0, 'num_object_copies': 19918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4028, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9959, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7950, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3918}, {'shards': '2', 'objects': 5932}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:46.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10390", 'reported_seq': 10479, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:40.278889+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:40.278889+0000', 'last_peered': '2024-09-11T01:18:40.278889+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:40.278889+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41664512, 'num_objects': 10172, 'num_object_clones': 0, 'num_object_copies': 20344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10172, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8027, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4071}, {'shards': '2', 'objects': 5995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10326", 'reported_seq': 12340, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:41.457449+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:41.457449+0000', 'last_peered': '2024-09-11T01:18:41.457449+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:41.457449+0000', 'last_undegraded': '2024-09-11T01:18:41.457449+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41066496, 'num_objects': 10026, 'num_object_clones': 0, 'num_object_copies': 20052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3220, 'num_objects_unfound': 0, 'num_objects_dirty': 10026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7016, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 944, 'num_bytes_recovered': 3866624, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6806}, {'shards': '0,2', 'objects': 3220}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'10475", 'reported_seq': 10612, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.428878+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:41.428878+0000', 'last_peered': '2024-09-11T01:18:41.428878+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:41.428878+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41660416, 'num_objects': 10171, 'num_object_clones': 0, 'num_object_copies': 20342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4190, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10171, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6868, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5984}, {'shards': '0,2', 'objects': 4044}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10437", 'reported_seq': 10572, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.374977+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:41.374977+0000', 'last_peered': '2024-09-11T01:18:41.374977+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:41.374977+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41627648, 'num_objects': 10163, 'num_object_clones': 0, 'num_object_copies': 20326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4147, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10163, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6972, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6018}, {'shards': '0,2', 'objects': 4013}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:46.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10218", 'reported_seq': 10347, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.418802+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:41.418802+0000', 'last_peered': '2024-09-11T01:18:41.418802+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:41.418802+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'200", 'ondisk_log_start': "32'200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40633286, 'num_objects': 9922, 'num_object_clones': 0, 'num_object_copies': 19844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9922, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6875, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5769}, {'shards': '0,2', 'objects': 4011}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:46.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:47.000 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:47.000 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41314/320520 objects degraded (12.890%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":327,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160260,"data_bytes":656376585,"bytes_used":2209726464,"bytes_avail":287700566016,"bytes_total":289910292480,"degraded_objects":41314,"degraded_total":320520,"degraded_ratio":0.12889679271184326,"misplaced_objects":3220,"misplaced_total":320520,"misplaced_ratio":0.010046174965680769,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":59718,"recovering_keys_per_sec":0,"num_objects_recovered":204,"num_bytes_recovered":716800,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":726604,"read_op_per_sec":0,"write_op_per_sec":410},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:47.025 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:47.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:47.578 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:47.578 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:47.602 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-11T01:18:46.493802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656376585,"num_objects":160260,"num_object_clones":0,"num_object_copies":320520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41314,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":160260,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":124985,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5100,"num_bytes_recovered":20759030,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2157936,"kb_used_data":1310452,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280957584,"statfs":{"total":289910292480,"available":287700566016,"internally_reserved":0,"allocated":1341902848,"data_stored":1341041279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":867748871},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,6,7,10,4,0,0,0,0,4,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2727912,"num_objects":-665,"num_object_clones":0,"num_object_copies":-1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-29,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":-665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4925,"num_write_kb":8517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002961"},"pg_stats":[{"pgid":"2.f","version":"41'10234","reported_seq":10310,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.427992+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:41.427992+0000","last_peered":"2024-09-11T01:18:41.427992+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:41.427992+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976384,"num_objects":10004,"num_object_clones":0,"num_object_copies":20008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10234,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5885},{"shards":"0,2","objects":4008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10521","reported_seq":10596,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:45.322587+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:45.322587+0000","last_peered":"2024-09-11T01:18:45.322587+0000","last_clean":"2024-09-11T01:18:45.322587+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:45.322587+0000","last_undegraded":"2024-09-11T01:18:45.322587+0000","last_fullsized":"2024-09-11T01:18:45.322587+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8619,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10267","reported_seq":10360,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:45.257491+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:45.257491+0000","last_peered":"2024-09-11T01:18:45.257491+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:45.257491+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40050712,"num_objects":9779,"num_object_clones":0,"num_object_copies":19558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8328,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3762},{"shards":"2","objects":5781}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10421","reported_seq":10461,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:45.165652+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:45.165652+0000","last_peered":"2024-09-11T01:18:45.165652+0000","last_clean":"2024-09-11T01:18:45.165652+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:45.165652+0000","last_undegraded":"2024-09-11T01:18:45.165652+0000","last_fullsized":"2024-09-11T01:18:45.165652+0000","mapping_epoch":31,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10421,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10273","reported_seq":10390,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.261360+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:41.261360+0000","last_peered":"2024-09-11T01:18:41.261360+0000","last_clean":"2024-09-11T01:18:41.261360+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:41.261360+0000","last_undegraded":"2024-09-11T01:18:41.261360+0000","last_fullsized":"2024-09-11T01:18:41.261360+0000","mapping_epoch":31,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40976867,"num_objects":10005,"num_object_clones":0,"num_object_copies":20010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10005,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7071,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10249","reported_seq":10383,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.426627+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:41.426627+0000","last_peered":"2024-09-11T01:18:41.426627+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:41.426627+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40788451,"num_objects":9959,"num_object_clones":0,"num_object_copies":19918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6737,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5846},{"shards":"0,2","objects":3971}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10538","reported_seq":10671,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.378508+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:41.378508+0000","last_peered":"2024-09-11T01:18:41.378508+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:41.378508+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42083270,"num_objects":10276,"num_object_clones":0,"num_object_copies":20552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4281,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10276,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7291,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5995},{"shards":"0,2","objects":4154}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10338","reported_seq":10453,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.403561+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:41.403561+0000","last_peered":"2024-09-11T01:18:41.403561+0000","last_clean":"2024-09-11T01:18:41.403561+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:41.403561+0000","last_undegraded":"2024-09-11T01:18:41.403561+0000","last_fullsized":"2024-09-11T01:18:41.403561+0000","mapping_epoch":31,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41312256,"num_objects":10086,"num_object_clones":0,"num_object_copies":20172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7291,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10337","reported_seq":18675,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.041014+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:44.041014+0000","last_peered":"2024-09-11T01:18:44.041014+0000","last_clean":"2024-09-11T01:18:44.041014+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:44.041014+0000","last_undegraded":"2024-09-11T01:18:44.041014+0000","last_fullsized":"2024-09-11T01:18:44.041014+0000","mapping_epoch":40,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40600035,"num_objects":9913,"num_object_clones":0,"num_object_copies":19826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8204,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10440","reported_seq":10572,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.388641+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:41.388641+0000","last_peered":"2024-09-11T01:18:41.388641+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:41.388641+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41590784,"num_objects":10154,"num_object_clones":0,"num_object_copies":20308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6804,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6053},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10315","reported_seq":10407,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:45.318046+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:45.318046+0000","last_peered":"2024-09-11T01:18:45.318046+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:45.318046+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40239587,"num_objects":9825,"num_object_clones":0,"num_object_copies":19650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8084,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3784},{"shards":"2","objects":5799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10531","reported_seq":10620,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:45.253172+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:45.253172+0000","last_peered":"2024-09-11T01:18:45.253172+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:45.253172+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41086976,"num_objects":10031,"num_object_clones":0,"num_object_copies":20062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8168,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3930},{"shards":"2","objects":5854}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10326","reported_seq":12340,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:41.457449+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:41.457449+0000","last_peered":"2024-09-11T01:18:41.457449+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:41.457449+0000","last_undegraded":"2024-09-11T01:18:41.457449+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41066496,"num_objects":10026,"num_object_clones":0,"num_object_copies":20052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":10026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7016,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6806},{"shards":"0,2","objects":3220}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10475","reported_seq":10612,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.428878+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:41.428878+0000","last_peered":"2024-09-11T01:18:41.428878+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:41.428878+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660416,"num_objects":10171,"num_object_clones":0,"num_object_copies":20342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6868,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5984},{"shards":"0,2","objects":4044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10437","reported_seq":10572,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.374977+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:41.374977+0000","last_peered":"2024-09-11T01:18:41.374977+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:41.374977+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41627648,"num_objects":10163,"num_object_clones":0,"num_object_copies":20326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6972,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6018},{"shards":"0,2","objects":4013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10218","reported_seq":10347,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.418802+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:41.418802+0000","last_peered":"2024-09-11T01:18:41.418802+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:41.418802+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'200","ondisk_log_start":"32'200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40633286,"num_objects":9922,"num_object_clones":0,"num_object_copies":19844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6875,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5769},{"shards":"0,2","objects":4011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656376566,"num_objects":160258,"num_object_clones":0,"num_object_copies":320516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41314,"num_objects_misplaced":3220,"num_objects_unfound":0,"num_objects_dirty":160258,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":124983,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5099,"num_bytes_recovered":20759011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1340227584,"data_stored":1340067694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160620,"ondisk_log_size":160620,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711836,"kb_used_data":451080,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93660004,"statfs":{"total":96636764160,"available":95907844096,"internally_reserved":0,"allocated":461905920,"data_stored":461634543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":266989230},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,1,2,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789994,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":562296,"kb_used_data":309864,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93809544,"statfs":{"total":96636764160,"available":96060973056,"internally_reserved":0,"allocated":317300736,"data_stored":316981926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":883804,"kb_used_data":549508,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93488036,"statfs":{"total":96636764160,"available":95731748864,"internally_reserved":0,"allocated":562696192,"data_stored":562424810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,4,3,1,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":562089984,"data_stored":562053854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316899328,"data_stored":316815770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461238272,"data_stored":461198070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:50.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:51.175 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:51.176 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:51.215 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-11T01:18:50.499270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647410441,"num_objects":158071,"num_object_clones":0,"num_object_copies":316142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41286,"num_objects_misplaced":3118,"num_objects_unfound":0,"num_objects_dirty":158071,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":127174,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5230,"num_bytes_recovered":21176822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146692,"kb_used_data":1299220,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280968828,"statfs":{"total":289910292480,"available":287712079872,"internally_reserved":0,"allocated":1330401280,"data_stored":1329539711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":867748871},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,5,5,7,7,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19587048,"num_objects":-4781,"num_object_clones":0,"num_object_copies":-9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-186,"num_objects_unfound":0,"num_objects_dirty":-4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5163,"num_write_kb":761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007434"},"pg_stats":[{"pgid":"2.f","version":"41'10369","reported_seq":10449,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.501957+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:46.501957+0000","last_peered":"2024-09-11T01:18:46.501957+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:46.501957+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40423424,"num_objects":9869,"num_object_clones":0,"num_object_copies":19738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10369,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5752},{"shards":"0,2","objects":3873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10674","reported_seq":10749,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.333405+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:50.333405+0000","last_peered":"2024-09-11T01:18:50.333405+0000","last_clean":"2024-09-11T01:18:50.333405+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:50.333405+0000","last_undegraded":"2024-09-11T01:18:50.333405+0000","last_fullsized":"2024-09-11T01:18:50.333405+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40338374,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8772,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10417","reported_seq":10520,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.364115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:50.364115+0000","last_peered":"2024-09-11T01:18:50.364115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:50.364115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436312,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8478,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3612},{"shards":"2","objects":5638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10566","reported_seq":10606,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.333955+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:50.333955+0000","last_peered":"2024-09-11T01:18:50.333955+0000","last_clean":"2024-09-11T01:18:50.333955+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:50.333955+0000","last_undegraded":"2024-09-11T01:18:50.333955+0000","last_fullsized":"2024-09-11T01:18:50.333955+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10417","reported_seq":10533,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.457549+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:46.457549+0000","last_peered":"2024-09-11T01:18:46.457549+0000","last_clean":"2024-09-11T01:18:46.457549+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:46.457549+0000","last_undegraded":"2024-09-11T01:18:46.457549+0000","last_fullsized":"2024-09-11T01:18:46.457549+0000","mapping_epoch":31,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40387043,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7215,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10372","reported_seq":10506,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.472590+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:46.472590+0000","last_peered":"2024-09-11T01:18:46.472590+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:46.472590+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284643,"num_objects":9836,"num_object_clones":0,"num_object_copies":19672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6860,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5723},{"shards":"0,2","objects":3848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10683","reported_seq":10820,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.448713+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:46.448713+0000","last_peered":"2024-09-11T01:18:46.448713+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:46.448713+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41489350,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7436,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5852},{"shards":"0,2","objects":4009}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10479","reported_seq":10594,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.472557+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:46.472557+0000","last_peered":"2024-09-11T01:18:46.472557+0000","last_clean":"2024-09-11T01:18:46.472557+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:46.472557+0000","last_undegraded":"2024-09-11T01:18:46.472557+0000","last_fullsized":"2024-09-11T01:18:46.472557+0000","mapping_epoch":31,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40734720,"num_objects":9945,"num_object_clones":0,"num_object_copies":19890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10479","reported_seq":18817,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.096203+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:49.096203+0000","last_peered":"2024-09-11T01:18:49.096203+0000","last_clean":"2024-09-11T01:18:49.096203+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:49.096203+0000","last_undegraded":"2024-09-11T01:18:49.096203+0000","last_fullsized":"2024-09-11T01:18:49.096203+0000","mapping_epoch":40,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8346,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10585","reported_seq":10716,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.483461+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:46.483461+0000","last_peered":"2024-09-11T01:18:46.483461+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:46.483461+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40996864,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6949,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5908},{"shards":"0,2","objects":3821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10444","reported_seq":10550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.288218+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:50.288218+0000","last_peered":"2024-09-11T01:18:50.288218+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:50.288218+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39711203,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8213,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3655},{"shards":"2","objects":5677}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10651","reported_seq":10748,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.287841+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:50.287841+0000","last_peered":"2024-09-11T01:18:50.287841+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:50.287841+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3810},{"shards":"2","objects":5738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10440","reported_seq":12658,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:46.463785+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:46.463785+0000","last_peered":"2024-09-11T01:18:46.463785+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:46.463785+0000","last_undegraded":"2024-09-11T01:18:46.463785+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40599552,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3118,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7130,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":4284416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6794},{"shards":"0,2","objects":3118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10613","reported_seq":10757,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.430708+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:46.430708+0000","last_peered":"2024-09-11T01:18:46.430708+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:46.430708+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7006,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5848},{"shards":"0,2","objects":3906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10573","reported_seq":10716,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.311501+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:46.311501+0000","last_peered":"2024-09-11T01:18:46.311501+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:46.311501+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41070592,"num_objects":10027,"num_object_clones":0,"num_object_copies":20054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7108,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5886},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10347","reported_seq":10479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.420025+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:46.420025+0000","last_peered":"2024-09-11T01:18:46.420025+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:46.420025+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40104902,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7004,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5640},{"shards":"0,2","objects":3882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647410422,"num_objects":158069,"num_object_clones":0,"num_object_copies":316138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41286,"num_objects_misplaced":3118,"num_objects_unfound":0,"num_objects_dirty":158069,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":127172,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5229,"num_bytes_recovered":21176803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328459776,"data_stored":1328299886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694972,"kb_used_data":446700,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93676868,"statfs":{"total":96636764160,"available":95925112832,"internally_reserved":0,"allocated":457420800,"data_stored":457149423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,2,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789995,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570152,"kb_used_data":309400,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93801688,"statfs":{"total":96636764160,"available":96052928512,"internally_reserved":0,"allocated":316825600,"data_stored":316506790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":266989231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881568,"kb_used_data":543120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93490272,"statfs":{"total":96636764160,"available":95734038528,"internally_reserved":0,"allocated":556154880,"data_stored":555883498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,7,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555491328,"data_stored":555455198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316317696,"data_stored":316234138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456650752,"data_stored":456610550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:51.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:51.785 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:51.786 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:51.813 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-11T01:18:50.499270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647410441,"num_objects":158071,"num_object_clones":0,"num_object_copies":316142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41286,"num_objects_misplaced":3118,"num_objects_unfound":0,"num_objects_dirty":158071,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":127174,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5230,"num_bytes_recovered":21176822,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160912,"ondisk_log_size":160912,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146692,"kb_used_data":1299220,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":280968828,"statfs":{"total":289910292480,"available":287712079872,"internally_reserved":0,"allocated":1330401280,"data_stored":1329539711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":867748871},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,5,5,7,7,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19587048,"num_objects":-4781,"num_object_clones":0,"num_object_copies":-9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-186,"num_objects_unfound":0,"num_objects_dirty":-4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5163,"num_write_kb":761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":761856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007434"},"pg_stats":[{"pgid":"2.f","version":"41'10369","reported_seq":10449,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.501957+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:46.501957+0000","last_peered":"2024-09-11T01:18:46.501957+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:46.501957+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40423424,"num_objects":9869,"num_object_clones":0,"num_object_copies":19738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10369,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5752},{"shards":"0,2","objects":3873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10674","reported_seq":10749,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.333405+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:50.333405+0000","last_peered":"2024-09-11T01:18:50.333405+0000","last_clean":"2024-09-11T01:18:50.333405+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:50.333405+0000","last_undegraded":"2024-09-11T01:18:50.333405+0000","last_fullsized":"2024-09-11T01:18:50.333405+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40338374,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8772,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10417","reported_seq":10520,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.364115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:50.364115+0000","last_peered":"2024-09-11T01:18:50.364115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:50.364115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436312,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8478,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3612},{"shards":"2","objects":5638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10566","reported_seq":10606,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.333955+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:50.333955+0000","last_peered":"2024-09-11T01:18:50.333955+0000","last_clean":"2024-09-11T01:18:50.333955+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:50.333955+0000","last_undegraded":"2024-09-11T01:18:50.333955+0000","last_fullsized":"2024-09-11T01:18:50.333955+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10417","reported_seq":10533,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.457549+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:46.457549+0000","last_peered":"2024-09-11T01:18:46.457549+0000","last_clean":"2024-09-11T01:18:46.457549+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:46.457549+0000","last_undegraded":"2024-09-11T01:18:46.457549+0000","last_fullsized":"2024-09-11T01:18:46.457549+0000","mapping_epoch":31,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40387043,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7215,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10372","reported_seq":10506,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.472590+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:46.472590+0000","last_peered":"2024-09-11T01:18:46.472590+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:46.472590+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40284643,"num_objects":9836,"num_object_clones":0,"num_object_copies":19672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6860,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5723},{"shards":"0,2","objects":3848}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10683","reported_seq":10820,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.448713+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:46.448713+0000","last_peered":"2024-09-11T01:18:46.448713+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:46.448713+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41489350,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7436,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5852},{"shards":"0,2","objects":4009}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10479","reported_seq":10594,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.472557+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:46.472557+0000","last_peered":"2024-09-11T01:18:46.472557+0000","last_clean":"2024-09-11T01:18:46.472557+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:46.472557+0000","last_undegraded":"2024-09-11T01:18:46.472557+0000","last_fullsized":"2024-09-11T01:18:46.472557+0000","mapping_epoch":31,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40734720,"num_objects":9945,"num_object_clones":0,"num_object_copies":19890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10479","reported_seq":18817,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.096203+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:49.096203+0000","last_peered":"2024-09-11T01:18:49.096203+0000","last_clean":"2024-09-11T01:18:49.096203+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:49.096203+0000","last_undegraded":"2024-09-11T01:18:49.096203+0000","last_fullsized":"2024-09-11T01:18:49.096203+0000","mapping_epoch":40,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8346,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10585","reported_seq":10716,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.483461+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:46.483461+0000","last_peered":"2024-09-11T01:18:46.483461+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:46.483461+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40996864,"num_objects":10009,"num_object_clones":0,"num_object_copies":20018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6949,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5908},{"shards":"0,2","objects":3821}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10444","reported_seq":10550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.288218+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:50.288218+0000","last_peered":"2024-09-11T01:18:50.288218+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:50.288218+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39711203,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8213,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3655},{"shards":"2","objects":5677}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10651","reported_seq":10748,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.287841+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:50.287841+0000","last_peered":"2024-09-11T01:18:50.287841+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:50.287841+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3810},{"shards":"2","objects":5738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10440","reported_seq":12658,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:46.463785+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:46.463785+0000","last_peered":"2024-09-11T01:18:46.463785+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:46.463785+0000","last_undegraded":"2024-09-11T01:18:46.463785+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40599552,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3118,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7130,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1046,"num_bytes_recovered":4284416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6794},{"shards":"0,2","objects":3118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10613","reported_seq":10757,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.430708+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:46.430708+0000","last_peered":"2024-09-11T01:18:46.430708+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:46.430708+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41095168,"num_objects":10033,"num_object_clones":0,"num_object_copies":20066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4188,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7006,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5848},{"shards":"0,2","objects":3906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10573","reported_seq":10716,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.311501+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:46.311501+0000","last_peered":"2024-09-11T01:18:46.311501+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:46.311501+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41070592,"num_objects":10027,"num_object_clones":0,"num_object_copies":20054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7108,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5886},{"shards":"0,2","objects":3877}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10347","reported_seq":10479,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.420025+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:46.420025+0000","last_peered":"2024-09-11T01:18:46.420025+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:46.420025+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'300","ondisk_log_start":"32'300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40104902,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7004,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5640},{"shards":"0,2","objects":3882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647410422,"num_objects":158069,"num_object_clones":0,"num_object_copies":316138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41286,"num_objects_misplaced":3118,"num_objects_unfound":0,"num_objects_dirty":158069,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":127172,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5229,"num_bytes_recovered":21176803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328459776,"data_stored":1328299886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694972,"kb_used_data":446700,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93676868,"statfs":{"total":96636764160,"available":95925112832,"internally_reserved":0,"allocated":457420800,"data_stored":457149423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,2,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789995,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570152,"kb_used_data":309400,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93801688,"statfs":{"total":96636764160,"available":96052928512,"internally_reserved":0,"allocated":316825600,"data_stored":316506790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":266989231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881568,"kb_used_data":543120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93490272,"statfs":{"total":96636764160,"available":95734038528,"internally_reserved":0,"allocated":556154880,"data_stored":555883498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,7,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":555491328,"data_stored":555455198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316317696,"data_stored":316234138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456650752,"data_stored":456610550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:51.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10369", 'reported_seq': 10449, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.501957+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:46.501957+0000', 'last_peered': '2024-09-11T01:18:46.501957+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:46.501957+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40423424, 'num_objects': 9869, 'num_object_clones': 0, 'num_object_copies': 19738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4117, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10369, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5752}, {'shards': '0,2', 'objects': 3873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10417", 'reported_seq': 10520, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:50.364115+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:50.364115+0000', 'last_peered': '2024-09-11T01:18:50.364115+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:50.364115+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39436312, 'num_objects': 9629, 'num_object_clones': 0, 'num_object_copies': 19258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3992, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9629, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8478, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3612}, {'shards': '2', 'objects': 5638}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'10372", 'reported_seq': 10506, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.472590+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:46.472590+0000', 'last_peered': '2024-09-11T01:18:46.472590+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:46.472590+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40284643, 'num_objects': 9836, 'num_object_clones': 0, 'num_object_copies': 19672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9836, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6860, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5723}, {'shards': '0,2', 'objects': 3848}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'10683", 'reported_seq': 10820, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.448713+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:46.448713+0000', 'last_peered': '2024-09-11T01:18:46.448713+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:46.448713+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'600", 'ondisk_log_start': "32'600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41489350, 'num_objects': 10131, 'num_object_clones': 0, 'num_object_copies': 20262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4279, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10131, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7436, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5852}, {'shards': '0,2', 'objects': 4009}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:51.815 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10585", 'reported_seq': 10716, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.483461+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:46.483461+0000', 'last_peered': '2024-09-11T01:18:46.483461+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:46.483461+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40996864, 'num_objects': 10009, 'num_object_clones': 0, 'num_object_copies': 20018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6949, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5908}, {'shards': '0,2', 'objects': 3821}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10444", 'reported_seq': 10550, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:50.288218+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:50.288218+0000', 'last_peered': '2024-09-11T01:18:50.288218+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:50.288218+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39711203, 'num_objects': 9696, 'num_object_clones': 0, 'num_object_copies': 19392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4020, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9696, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8213, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3655}, {'shards': '2', 'objects': 5677}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10651", 'reported_seq': 10748, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:50.287841+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:50.287841+0000', 'last_peered': '2024-09-11T01:18:50.287841+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:50.287841+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'600", 'ondisk_log_start': "32'600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40595456, 'num_objects': 9911, 'num_object_clones': 0, 'num_object_copies': 19822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4175, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9911, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8288, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3810}, {'shards': '2', 'objects': 5738}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10440", 'reported_seq': 12658, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:46.463785+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:46.463785+0000', 'last_peered': '2024-09-11T01:18:46.463785+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:46.463785+0000', 'last_undegraded': '2024-09-11T01:18:46.463785+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40599552, 'num_objects': 9912, 'num_object_clones': 0, 'num_object_copies': 19824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3118, 'num_objects_unfound': 0, 'num_objects_dirty': 9912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7130, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1046, 'num_bytes_recovered': 4284416, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6794}, {'shards': '0,2', 'objects': 3118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'10613", 'reported_seq': 10757, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.430708+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:46.430708+0000', 'last_peered': '2024-09-11T01:18:46.430708+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:46.430708+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'600", 'ondisk_log_start': "32'600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41095168, 'num_objects': 10033, 'num_object_clones': 0, 'num_object_copies': 20066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4188, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7006, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5848}, {'shards': '0,2', 'objects': 3906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.816 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:51.817 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10573", 'reported_seq': 10716, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.311501+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:46.311501+0000', 'last_peered': '2024-09-11T01:18:46.311501+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:46.311501+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41070592, 'num_objects': 10027, 'num_object_clones': 0, 'num_object_copies': 20054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4143, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10027, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7108, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5886}, {'shards': '0,2', 'objects': 3877}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.817 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:51.817 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10347", 'reported_seq': 10479, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.420025+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:46.420025+0000', 'last_peered': '2024-09-11T01:18:46.420025+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:46.420025+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'300", 'ondisk_log_start': "32'300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40104902, 'num_objects': 9793, 'num_object_clones': 0, 'num_object_copies': 19586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9793, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7004, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5640}, {'shards': '0,2', 'objects': 3882}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:52.511 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:52.511 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41286/316142 objects degraded (13.059%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":332,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158071,"data_bytes":647410441,"bytes_used":2198212608,"bytes_avail":287712079872,"bytes_total":289910292480,"degraded_objects":41286,"degraded_total":316142,"degraded_ratio":0.13059321444161168,"misplaced_objects":3118,"misplaced_total":316142,"misplaced_ratio":0.0098626566542882634,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":63448,"recovering_keys_per_sec":0,"num_objects_recovered":243,"num_bytes_recovered":761856,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":64898,"read_op_per_sec":0,"write_op_per_sec":429},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:52.546 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:52.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:53.114 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:53.115 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:53.135 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-11T01:18:52.502063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642040585,"num_objects":156760,"num_object_clones":0,"num_object_copies":313520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":156760,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":128485,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5344,"num_bytes_recovered":21606902,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2144516,"kb_used_data":1292884,"kb_used_omap":12,"kb_used_meta":851571,"kb_avail":280971004,"statfs":{"total":289910292480,"available":287714308096,"internally_reserved":0,"allocated":1323913216,"data_stored":1323051647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":872008711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,5,9,8,3,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23199744,"num_objects":-5664,"num_object_clones":0,"num_object_copies":-11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-62,"num_objects_misplaced":-291,"num_objects_unfound":0,"num_objects_dirty":-5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5664,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":353,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009730"},"pg_stats":[{"pgid":"2.f","version":"41'10506","reported_seq":10587,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.494879+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:51.494879+0000","last_peered":"2024-09-11T01:18:51.494879+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:51.494879+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39862272,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10506,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":3736}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10674","reported_seq":10749,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.333405+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:50.333405+0000","last_peered":"2024-09-11T01:18:50.333405+0000","last_clean":"2024-09-11T01:18:50.333405+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:50.333405+0000","last_undegraded":"2024-09-11T01:18:50.333405+0000","last_fullsized":"2024-09-11T01:18:50.333405+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40338374,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8772,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10417","reported_seq":10520,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.364115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:50.364115+0000","last_peered":"2024-09-11T01:18:50.364115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:50.364115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39436312,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3992,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8478,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3612},{"shards":"2","objects":5638}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10566","reported_seq":10606,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.333955+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:50.333955+0000","last_peered":"2024-09-11T01:18:50.333955+0000","last_clean":"2024-09-11T01:18:50.333955+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:50.333955+0000","last_undegraded":"2024-09-11T01:18:50.333955+0000","last_fullsized":"2024-09-11T01:18:50.333955+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40124416,"num_objects":9796,"num_object_clones":0,"num_object_copies":19592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10566,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10549","reported_seq":10666,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.529101+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:51.529101+0000","last_peered":"2024-09-11T01:18:51.529101+0000","last_clean":"2024-09-11T01:18:51.529101+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:51.529101+0000","last_undegraded":"2024-09-11T01:18:51.529101+0000","last_fullsized":"2024-09-11T01:18:51.529101+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846371,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7347,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10496","reported_seq":10636,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.504414+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:51.504414+0000","last_peered":"2024-09-11T01:18:51.504414+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:51.504414+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39776739,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6984,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5602},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10811","reported_seq":10952,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.432391+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:51.432391+0000","last_peered":"2024-09-11T01:18:51.432391+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:51.432391+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7564,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5726},{"shards":"0,2","objects":3881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10618","reported_seq":10733,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.477965+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:51.477965+0000","last_peered":"2024-09-11T01:18:51.477965+0000","last_clean":"2024-09-11T01:18:51.477965+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:51.477965+0000","last_undegraded":"2024-09-11T01:18:51.477965+0000","last_fullsized":"2024-09-11T01:18:51.477965+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165376,"num_objects":9806,"num_object_clones":0,"num_object_copies":19612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10479","reported_seq":18817,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.096203+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:49.096203+0000","last_peered":"2024-09-11T01:18:49.096203+0000","last_clean":"2024-09-11T01:18:49.096203+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:49.096203+0000","last_undegraded":"2024-09-11T01:18:49.096203+0000","last_fullsized":"2024-09-11T01:18:49.096203+0000","mapping_epoch":40,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8346,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10727","reported_seq":10860,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.545429+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:51.545429+0000","last_peered":"2024-09-11T01:18:51.545429+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:51.545429+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40415232,"num_objects":9867,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7091,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10444","reported_seq":10550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.288218+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:50.288218+0000","last_peered":"2024-09-11T01:18:50.288218+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:50.288218+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39711203,"num_objects":9696,"num_object_clones":0,"num_object_copies":19392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8213,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3655},{"shards":"2","objects":5677}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10651","reported_seq":10748,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:50.287841+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:50.287841+0000","last_peered":"2024-09-11T01:18:50.287841+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:50.287841+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40595456,"num_objects":9911,"num_object_clones":0,"num_object_copies":19822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3810},{"shards":"2","objects":5738}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10559","reported_seq":12986,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:51.525404+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:51.525404+0000","last_peered":"2024-09-11T01:18:51.525404+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:51.525404+0000","last_undegraded":"2024-09-11T01:18:51.525404+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1151,"num_bytes_recovered":4714496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6780},{"shards":"0,2","objects":3013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10757","reported_seq":10903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.535257+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:51.535257+0000","last_peered":"2024-09-11T01:18:51.535257+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:51.535257+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40505344,"num_objects":9889,"num_object_clones":0,"num_object_copies":19778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7150,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5705},{"shards":"0,2","objects":3762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10692","reported_seq":10835,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.555891+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:51.555891+0000","last_peered":"2024-09-11T01:18:51.555891+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:51.555891+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40583168,"num_objects":9908,"num_object_clones":0,"num_object_copies":19816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10474","reported_seq":10610,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.481233+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:51.481233+0000","last_peered":"2024-09-11T01:18:51.481233+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:51.481233+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584710,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7131,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5515},{"shards":"0,2","objects":3755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642040566,"num_objects":156758,"num_object_clones":0,"num_object_copies":313516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41277,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":156758,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":128483,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5343,"num_bytes_recovered":21606883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1321873408,"data_stored":1321713518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":694972,"kb_used_data":446700,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93676868,"statfs":{"total":96636764160,"available":95925112832,"internally_reserved":0,"allocated":457420800,"data_stored":457149423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":254209710},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,2,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789995,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570152,"kb_used_data":309400,"kb_used_omap":4,"kb_used_meta":260731,"kb_avail":93801688,"statfs":{"total":96636764160,"available":96052928512,"internally_reserved":0,"allocated":316825600,"data_stored":316506790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":266989231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879392,"kb_used_data":536784,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93492448,"statfs":{"total":96636764160,"available":95736266752,"internally_reserved":0,"allocated":549666816,"data_stored":549395434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,3,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548904960,"data_stored":548868830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316317696,"data_stored":316234138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456650752,"data_stored":456610550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:56.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:56.700 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:56.701 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:56.728 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-11T01:18:56.503028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638870281,"num_objects":155986,"num_object_clones":0,"num_object_copies":311972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41263,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":155986,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":129259,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5358,"num_bytes_recovered":21606902,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2149660,"kb_used_data":1288292,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":280965860,"statfs":{"total":289910292480,"available":287709040640,"internally_reserved":0,"allocated":1319211008,"data_stored":1318349439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":881970183},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,9,7,4,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20275200,"num_objects":-4950,"num_object_clones":0,"num_object_copies":-9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-55,"num_objects_misplaced":-207,"num_objects_unfound":0,"num_objects_dirty":-4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":847872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009704"},"pg_stats":[{"pgid":"2.f","version":"41'10506","reported_seq":10587,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.494879+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:51.494879+0000","last_peered":"2024-09-11T01:18:51.494879+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:51.494879+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39862272,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10506,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":3736}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10800","reported_seq":10875,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.368193+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:55.368193+0000","last_peered":"2024-09-11T01:18:55.368193+0000","last_clean":"2024-09-11T01:18:55.368193+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:55.368193+0000","last_undegraded":"2024-09-11T01:18:55.368193+0000","last_fullsized":"2024-09-11T01:18:55.368193+0000","mapping_epoch":31,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39822278,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8898,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10535","reported_seq":10650,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.433673+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:55.433673+0000","last_peered":"2024-09-11T01:18:55.433673+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:55.433673+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952984,"num_objects":9511,"num_object_clones":0,"num_object_copies":19022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8596,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5524}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10697","reported_seq":10737,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.430995+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:55.430995+0000","last_peered":"2024-09-11T01:18:55.430995+0000","last_clean":"2024-09-11T01:18:55.430995+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:55.430995+0000","last_undegraded":"2024-09-11T01:18:55.430995+0000","last_fullsized":"2024-09-11T01:18:55.430995+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39587840,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10549","reported_seq":10666,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.529101+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:51.529101+0000","last_peered":"2024-09-11T01:18:51.529101+0000","last_clean":"2024-09-11T01:18:51.529101+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:51.529101+0000","last_undegraded":"2024-09-11T01:18:51.529101+0000","last_fullsized":"2024-09-11T01:18:51.529101+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846371,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7347,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10496","reported_seq":10636,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.504414+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:51.504414+0000","last_peered":"2024-09-11T01:18:51.504414+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:51.504414+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39776739,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6984,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5602},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10811","reported_seq":10952,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.432391+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:51.432391+0000","last_peered":"2024-09-11T01:18:51.432391+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:51.432391+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7564,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5726},{"shards":"0,2","objects":3881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10618","reported_seq":10733,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.477965+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:51.477965+0000","last_peered":"2024-09-11T01:18:51.477965+0000","last_clean":"2024-09-11T01:18:51.477965+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:51.477965+0000","last_undegraded":"2024-09-11T01:18:51.477965+0000","last_fullsized":"2024-09-11T01:18:51.477965+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165376,"num_objects":9806,"num_object_clones":0,"num_object_copies":19612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10605","reported_seq":18943,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.216601+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:54.216601+0000","last_peered":"2024-09-11T01:18:54.216601+0000","last_clean":"2024-09-11T01:18:54.216601+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:54.216601+0000","last_undegraded":"2024-09-11T01:18:54.216601+0000","last_fullsized":"2024-09-11T01:18:54.216601+0000","mapping_epoch":40,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39502307,"num_objects":9645,"num_object_clones":0,"num_object_copies":19290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8472,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10727","reported_seq":10860,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.545429+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:51.545429+0000","last_peered":"2024-09-11T01:18:51.545429+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:51.545429+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40415232,"num_objects":9867,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7091,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10576","reported_seq":10686,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.378303+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:55.378303+0000","last_peered":"2024-09-11T01:18:55.378303+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:55.378303+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39170531,"num_objects":9564,"num_object_clones":0,"num_object_copies":19128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8345,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3523},{"shards":"2","objects":5547}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10792","reported_seq":10905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.312746+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:55.312746+0000","last_peered":"2024-09-11T01:18:55.312746+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:55.312746+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40017920,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8429,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3669},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10559","reported_seq":12986,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:51.525404+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:51.525404+0000","last_peered":"2024-09-11T01:18:51.525404+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:51.525404+0000","last_undegraded":"2024-09-11T01:18:51.525404+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1151,"num_bytes_recovered":4714496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6780},{"shards":"0,2","objects":3013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10757","reported_seq":10903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.535257+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:51.535257+0000","last_peered":"2024-09-11T01:18:51.535257+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:51.535257+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40505344,"num_objects":9889,"num_object_clones":0,"num_object_copies":19778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7150,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5705},{"shards":"0,2","objects":3762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10692","reported_seq":10835,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.555891+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:51.555891+0000","last_peered":"2024-09-11T01:18:51.555891+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:51.555891+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40583168,"num_objects":9908,"num_object_clones":0,"num_object_copies":19816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10474","reported_seq":10610,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.481233+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:51.481233+0000","last_peered":"2024-09-11T01:18:51.481233+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:51.481233+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584710,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7131,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5515},{"shards":"0,2","objects":3755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638870262,"num_objects":155984,"num_object_clones":0,"num_object_copies":311968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41263,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":155984,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":129257,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5357,"num_bytes_recovered":21606883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317199872,"data_stored":1317039982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713228,"kb_used_data":442740,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658612,"statfs":{"total":96636764160,"available":95906418688,"internally_reserved":0,"allocated":453365760,"data_stored":453094383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557040,"kb_used_data":308768,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93814800,"statfs":{"total":96636764160,"available":96066355200,"internally_reserved":0,"allocated":316178432,"data_stored":315859622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879392,"kb_used_data":536784,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93492448,"statfs":{"total":96636764160,"available":95736266752,"internally_reserved":0,"allocated":549666816,"data_stored":549395434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,3,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548904960,"data_stored":548868830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315826176,"data_stored":315742618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452468736,"data_stored":452428534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:56.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:57.317 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:57.320 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:57.340 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-11T01:18:56.503028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638870281,"num_objects":155986,"num_object_clones":0,"num_object_copies":311972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41263,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":155986,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":129259,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5358,"num_bytes_recovered":21606902,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2149660,"kb_used_data":1288292,"kb_used_omap":12,"kb_used_meta":861299,"kb_avail":280965860,"statfs":{"total":289910292480,"available":287709040640,"internally_reserved":0,"allocated":1319211008,"data_stored":1318349439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":881970183},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,9,7,4,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20275200,"num_objects":-4950,"num_object_clones":0,"num_object_copies":-9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-55,"num_objects_misplaced":-207,"num_objects_unfound":0,"num_objects_dirty":-4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":847872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009704"},"pg_stats":[{"pgid":"2.f","version":"41'10506","reported_seq":10587,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.494879+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:51.494879+0000","last_peered":"2024-09-11T01:18:51.494879+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:51.494879+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39862272,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10506,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5615},{"shards":"0,2","objects":3736}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10800","reported_seq":10875,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.368193+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:55.368193+0000","last_peered":"2024-09-11T01:18:55.368193+0000","last_clean":"2024-09-11T01:18:55.368193+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:55.368193+0000","last_undegraded":"2024-09-11T01:18:55.368193+0000","last_fullsized":"2024-09-11T01:18:55.368193+0000","mapping_epoch":31,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39822278,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8898,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10535","reported_seq":10650,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.433673+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:55.433673+0000","last_peered":"2024-09-11T01:18:55.433673+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:55.433673+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952984,"num_objects":9511,"num_object_clones":0,"num_object_copies":19022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8596,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5524}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10697","reported_seq":10737,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.430995+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:55.430995+0000","last_peered":"2024-09-11T01:18:55.430995+0000","last_clean":"2024-09-11T01:18:55.430995+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:55.430995+0000","last_undegraded":"2024-09-11T01:18:55.430995+0000","last_fullsized":"2024-09-11T01:18:55.430995+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39587840,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10549","reported_seq":10666,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.529101+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:51.529101+0000","last_peered":"2024-09-11T01:18:51.529101+0000","last_clean":"2024-09-11T01:18:51.529101+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:51.529101+0000","last_undegraded":"2024-09-11T01:18:51.529101+0000","last_fullsized":"2024-09-11T01:18:51.529101+0000","mapping_epoch":31,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846371,"num_objects":9729,"num_object_clones":0,"num_object_copies":19458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7347,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10496","reported_seq":10636,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.504414+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:51.504414+0000","last_peered":"2024-09-11T01:18:51.504414+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:51.504414+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39776739,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6984,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5602},{"shards":"0,2","objects":3724}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10811","reported_seq":10952,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.432391+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:51.432391+0000","last_peered":"2024-09-11T01:18:51.432391+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:51.432391+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7564,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5726},{"shards":"0,2","objects":3881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10618","reported_seq":10733,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.477965+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:51.477965+0000","last_peered":"2024-09-11T01:18:51.477965+0000","last_clean":"2024-09-11T01:18:51.477965+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:51.477965+0000","last_undegraded":"2024-09-11T01:18:51.477965+0000","last_fullsized":"2024-09-11T01:18:51.477965+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40165376,"num_objects":9806,"num_object_clones":0,"num_object_copies":19612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10605","reported_seq":18943,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.216601+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:54.216601+0000","last_peered":"2024-09-11T01:18:54.216601+0000","last_clean":"2024-09-11T01:18:54.216601+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:54.216601+0000","last_undegraded":"2024-09-11T01:18:54.216601+0000","last_fullsized":"2024-09-11T01:18:54.216601+0000","mapping_epoch":40,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39502307,"num_objects":9645,"num_object_clones":0,"num_object_copies":19290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8472,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10727","reported_seq":10860,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.545429+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:51.545429+0000","last_peered":"2024-09-11T01:18:51.545429+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:51.545429+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40415232,"num_objects":9867,"num_object_clones":0,"num_object_copies":19734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7091,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10576","reported_seq":10686,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.378303+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:55.378303+0000","last_peered":"2024-09-11T01:18:55.378303+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:55.378303+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39170531,"num_objects":9564,"num_object_clones":0,"num_object_copies":19128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8345,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3523},{"shards":"2","objects":5547}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10792","reported_seq":10905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.312746+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:55.312746+0000","last_peered":"2024-09-11T01:18:55.312746+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:55.312746+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40017920,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8429,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3669},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10559","reported_seq":12986,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:51.525404+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:51.525404+0000","last_peered":"2024-09-11T01:18:51.525404+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:51.525404+0000","last_undegraded":"2024-09-11T01:18:51.525404+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7249,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1151,"num_bytes_recovered":4714496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6780},{"shards":"0,2","objects":3013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10757","reported_seq":10903,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.535257+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:51.535257+0000","last_peered":"2024-09-11T01:18:51.535257+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:51.535257+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40505344,"num_objects":9889,"num_object_clones":0,"num_object_copies":19778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7150,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5705},{"shards":"0,2","objects":3762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10692","reported_seq":10835,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.555891+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:51.555891+0000","last_peered":"2024-09-11T01:18:51.555891+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:51.555891+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40583168,"num_objects":9908,"num_object_clones":0,"num_object_copies":19816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4143,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7227,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10474","reported_seq":10610,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.481233+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:51.481233+0000","last_peered":"2024-09-11T01:18:51.481233+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:51.481233+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'400","ondisk_log_start":"32'400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39584710,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7131,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5515},{"shards":"0,2","objects":3755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638870262,"num_objects":155984,"num_object_clones":0,"num_object_copies":311968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41263,"num_objects_misplaced":3013,"num_objects_unfound":0,"num_objects_dirty":155984,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":129257,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5357,"num_bytes_recovered":21606883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1317199872,"data_stored":1317039982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713228,"kb_used_data":442740,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658612,"statfs":{"total":96636764160,"available":95906418688,"internally_reserved":0,"allocated":453365760,"data_stored":453094383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557040,"kb_used_data":308768,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93814800,"statfs":{"total":96636764160,"available":96066355200,"internally_reserved":0,"allocated":316178432,"data_stored":315859622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879392,"kb_used_data":536784,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93492448,"statfs":{"total":96636764160,"available":95736266752,"internally_reserved":0,"allocated":549666816,"data_stored":549395434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,3,0,0,0,0,1],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":548904960,"data_stored":548868830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315826176,"data_stored":315742618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452468736,"data_stored":452428534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:57.341 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:57.341 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10506", 'reported_seq': 10587, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.494879+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:18:51.494879+0000', 'last_peered': '2024-09-11T01:18:51.494879+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:18:51.494879+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39862272, 'num_objects': 9732, 'num_object_clones': 0, 'num_object_copies': 19464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4117, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10506, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5615}, {'shards': '0,2', 'objects': 3736}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.341 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10535", 'reported_seq': 10650, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:55.433673+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:18:55.433673+0000', 'last_peered': '2024-09-11T01:18:55.433673+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:18:55.433673+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38952984, 'num_objects': 9511, 'num_object_clones': 0, 'num_object_copies': 19022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9511, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8596, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3494}, {'shards': '2', 'objects': 5524}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'10496", 'reported_seq': 10636, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.504414+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:18:51.504414+0000', 'last_peered': '2024-09-11T01:18:51.504414+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:18:51.504414+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39776739, 'num_objects': 9712, 'num_object_clones': 0, 'num_object_copies': 19424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4111, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9712, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6984, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5602}, {'shards': '0,2', 'objects': 3724}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'10811", 'reported_seq': 10952, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.432391+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:18:51.432391+0000', 'last_peered': '2024-09-11T01:18:51.432391+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:18:51.432391+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'800", 'ondisk_log_start': "32'800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40965062, 'num_objects': 10003, 'num_object_clones': 0, 'num_object_copies': 20006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4277, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10003, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7564, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 6, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5726}, {'shards': '0,2', 'objects': 3881}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10727", 'reported_seq': 10860, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.545429+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:18:51.545429+0000', 'last_peered': '2024-09-11T01:18:51.545429+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:18:51.545429+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40415232, 'num_objects': 9867, 'num_object_clones': 0, 'num_object_copies': 19734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4102, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7091, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 5, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5767}, {'shards': '0,2', 'objects': 3679}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10576", 'reported_seq': 10686, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:55.378303+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:18:55.378303+0000', 'last_peered': '2024-09-11T01:18:55.378303+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:18:55.378303+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39170531, 'num_objects': 9564, 'num_object_clones': 0, 'num_object_copies': 19128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4018, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9564, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8345, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3523}, {'shards': '2', 'objects': 5547}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:18:57.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10792", 'reported_seq': 10905, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:55.312746+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:18:55.312746+0000', 'last_peered': '2024-09-11T01:18:55.312746+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:18:55.312746+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40017920, 'num_objects': 9770, 'num_object_clones': 0, 'num_object_copies': 19540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4167, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8429, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3669}, {'shards': '2', 'objects': 5605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10559", 'reported_seq': 12986, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:18:51.525404+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:18:51.525404+0000', 'last_peered': '2024-09-11T01:18:51.525404+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:18:51.525404+0000', 'last_undegraded': '2024-09-11T01:18:51.525404+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'500", 'ondisk_log_start': "32'500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40112128, 'num_objects': 9793, 'num_object_clones': 0, 'num_object_copies': 19586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3013, 'num_objects_unfound': 0, 'num_objects_dirty': 9793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7249, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1151, 'num_bytes_recovered': 4714496, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6780}, {'shards': '0,2', 'objects': 3013}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'10757", 'reported_seq': 10903, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.535257+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:18:51.535257+0000', 'last_peered': '2024-09-11T01:18:51.535257+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:18:51.535257+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40505344, 'num_objects': 9889, 'num_object_clones': 0, 'num_object_copies': 19778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4187, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7150, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 8, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5705}, {'shards': '0,2', 'objects': 3762}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10692", 'reported_seq': 10835, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.555891+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:18:51.555891+0000', 'last_peered': '2024-09-11T01:18:51.555891+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:18:51.555891+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'600", 'ondisk_log_start': "32'600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40583168, 'num_objects': 9908, 'num_object_clones': 0, 'num_object_copies': 19816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4143, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9908, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7227, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5767}, {'shards': '0,2', 'objects': 3758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:57.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10474", 'reported_seq': 10610, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.481233+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:18:51.481233+0000', 'last_peered': '2024-09-11T01:18:51.481233+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:18:51.481233+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'400", 'ondisk_log_start': "32'400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39584710, 'num_objects': 9666, 'num_object_clones': 0, 'num_object_copies': 19332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9666, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7131, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5515}, {'shards': '0,2', 'objects': 3755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:57.344 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:58.058 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:58.058 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41263/311972 objects degraded (13.227%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":338,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":155986,"data_bytes":638870281,"bytes_used":2201251840,"bytes_avail":287709040640,"bytes_total":289910292480,"degraded_objects":41263,"degraded_total":311972,"degraded_ratio":0.13226507507083968,"misplaced_objects":3013,"misplaced_total":311972,"misplaced_ratio":0.009657918018283692,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":70598,"recovering_keys_per_sec":0,"num_objects_recovered":262,"num_bytes_recovered":847872,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":412},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:58.087 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:58.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:58.682 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:18:58.682 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:18:58.707 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-11T01:18:58.503505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633582345,"num_objects":154695,"num_object_clones":0,"num_object_copies":309390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41231,"num_objects_misplaced":2882,"num_objects_unfound":0,"num_objects_dirty":154695,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":130550,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5521,"num_bytes_recovered":22143478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2152048,"kb_used_data":1282356,"kb_used_omap":12,"kb_used_meta":869619,"kb_avail":280963472,"statfs":{"total":289910292480,"available":287706595328,"internally_reserved":0,"allocated":1313132544,"data_stored":1312270975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":890489863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,6,4,2,0,0,0,0,7,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22794240,"num_objects":-5565,"num_object_clones":0,"num_object_copies":-11130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-83,"num_objects_misplaced":-338,"num_objects_unfound":0,"num_objects_dirty":-5565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5565,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":421,"num_bytes_recovered":1384448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009703"},"pg_stats":[{"pgid":"2.f","version":"41'10652","reported_seq":10747,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.527652+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:56.527652+0000","last_peered":"2024-09-11T01:18:56.527652+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:56.527652+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39264256,"num_objects":9586,"num_object_clones":0,"num_object_copies":19172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10652,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5476},{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10800","reported_seq":10875,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.368193+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:18:55.368193+0000","last_peered":"2024-09-11T01:18:55.368193+0000","last_clean":"2024-09-11T01:18:55.368193+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:18:55.368193+0000","last_undegraded":"2024-09-11T01:18:55.368193+0000","last_fullsized":"2024-09-11T01:18:55.368193+0000","mapping_epoch":31,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39822278,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8898,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10535","reported_seq":10650,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.433673+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:18:55.433673+0000","last_peered":"2024-09-11T01:18:55.433673+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:18:55.433673+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38952984,"num_objects":9511,"num_object_clones":0,"num_object_copies":19022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8596,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3494},{"shards":"2","objects":5524}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10697","reported_seq":10737,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.430995+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:18:55.430995+0000","last_peered":"2024-09-11T01:18:55.430995+0000","last_clean":"2024-09-11T01:18:55.430995+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:18:55.430995+0000","last_undegraded":"2024-09-11T01:18:55.430995+0000","last_fullsized":"2024-09-11T01:18:55.430995+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39587840,"num_objects":9665,"num_object_clones":0,"num_object_copies":19330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10678","reported_seq":10795,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:56.539774+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:56.539774+0000","last_peered":"2024-09-11T01:18:56.539774+0000","last_clean":"2024-09-11T01:18:56.539774+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:56.539774+0000","last_undegraded":"2024-09-11T01:18:56.539774+0000","last_fullsized":"2024-09-11T01:18:56.539774+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317987,"num_objects":9600,"num_object_clones":0,"num_object_copies":19200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7476,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10642","reported_seq":10794,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.408950+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:56.408950+0000","last_peered":"2024-09-11T01:18:56.408950+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:56.408950+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178723,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7130,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5462},{"shards":"0,2","objects":3578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10954","reported_seq":11103,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.565857+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:56.565857+0000","last_peered":"2024-09-11T01:18:56.565857+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:56.565857+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40379334,"num_objects":9860,"num_object_clones":0,"num_object_copies":19720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9860,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7707,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5587},{"shards":"0,2","objects":3738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10737","reported_seq":10852,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:56.600842+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:56.600842+0000","last_peered":"2024-09-11T01:18:56.600842+0000","last_clean":"2024-09-11T01:18:56.600842+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:56.600842+0000","last_undegraded":"2024-09-11T01:18:56.600842+0000","last_fullsized":"2024-09-11T01:18:56.600842+0000","mapping_epoch":31,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39677952,"num_objects":9687,"num_object_clones":0,"num_object_copies":19374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7690,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10605","reported_seq":18943,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.216601+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:54.216601+0000","last_peered":"2024-09-11T01:18:54.216601+0000","last_clean":"2024-09-11T01:18:54.216601+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:54.216601+0000","last_undegraded":"2024-09-11T01:18:54.216601+0000","last_fullsized":"2024-09-11T01:18:54.216601+0000","mapping_epoch":40,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39502307,"num_objects":9645,"num_object_clones":0,"num_object_copies":19290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8472,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10870","reported_seq":11016,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.516659+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:56.516659+0000","last_peered":"2024-09-11T01:18:56.516659+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:56.516659+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39829504,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4096,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7234,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":3536}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10576","reported_seq":10686,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.378303+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:18:55.378303+0000","last_peered":"2024-09-11T01:18:55.378303+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:18:55.378303+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'500","ondisk_log_start":"32'500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39170531,"num_objects":9564,"num_object_clones":0,"num_object_copies":19128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9564,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8345,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3523},{"shards":"2","objects":5547}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10792","reported_seq":10905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:55.312746+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:18:55.312746+0000","last_peered":"2024-09-11T01:18:55.312746+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:18:55.312746+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40017920,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4167,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8429,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3669},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10661","reported_seq":13351,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:56.593232+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:56.593232+0000","last_peered":"2024-09-11T01:18:56.593232+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:56.593232+0000","last_undegraded":"2024-09-11T01:18:56.593232+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2882,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7351,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":5251072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6809},{"shards":"0,2","objects":2882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10876","reported_seq":11034,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.610296+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:56.610296+0000","last_peered":"2024-09-11T01:18:56.610296+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:56.610296+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40017920,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5592},{"shards":"0,2","objects":3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10807","reported_seq":10956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.496198+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:56.496198+0000","last_peered":"2024-09-11T01:18:56.496198+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:56.496198+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7342,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5655},{"shards":"0,2","objects":3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10603","reported_seq":10739,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.503250+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:56.503250+0000","last_peered":"2024-09-11T01:18:56.503250+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:56.503250+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39056326,"num_objects":9537,"num_object_clones":0,"num_object_copies":19074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7260,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5386},{"shards":"0,2","objects":3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633582326,"num_objects":154693,"num_object_clones":0,"num_object_copies":309386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41231,"num_objects_misplaced":2882,"num_objects_unfound":0,"num_objects_dirty":154693,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":130548,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5520,"num_bytes_recovered":22143459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1310957568,"data_stored":1310797678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713228,"kb_used_data":442740,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93658612,"statfs":{"total":96636764160,"available":95906418688,"internally_reserved":0,"allocated":453365760,"data_stored":453094383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,0,0,0,0,0,0,7],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557040,"kb_used_data":308768,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93814800,"statfs":{"total":96636764160,"available":96066355200,"internally_reserved":0,"allocated":316178432,"data_stored":315859622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881780,"kb_used_data":530848,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93490060,"statfs":{"total":96636764160,"available":95733821440,"internally_reserved":0,"allocated":543588352,"data_stored":543316970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,1,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":542662656,"data_stored":542626526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315826176,"data_stored":315742618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452468736,"data_stored":452428534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:01.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:02.259 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:02.261 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:02.284 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-11T01:19:00.503999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630739721,"num_objects":154001,"num_object_clones":0,"num_object_copies":308002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41226,"num_objects_misplaced":2882,"num_objects_unfound":0,"num_objects_dirty":154001,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":131244,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5526,"num_bytes_recovered":22143478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2188200,"kb_used_data":1278252,"kb_used_omap":12,"kb_used_meta":909875,"kb_avail":280927320,"statfs":{"total":289910292480,"available":287669575680,"internally_reserved":0,"allocated":1308930048,"data_stored":1308068479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":931712007},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,1,6,1,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20107264,"num_objects":-4909,"num_object_clones":0,"num_object_copies":-9818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-78,"num_objects_misplaced":-236,"num_objects_unfound":0,"num_objects_dirty":-4909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4909,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006496"},"pg_stats":[{"pgid":"2.f","version":"41'10652","reported_seq":10747,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.527652+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:18:56.527652+0000","last_peered":"2024-09-11T01:18:56.527652+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:18:56.527652+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39264256,"num_objects":9586,"num_object_clones":0,"num_object_copies":19172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10652,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5476},{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10926","reported_seq":11002,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.440753+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:00.440753+0000","last_peered":"2024-09-11T01:19:00.440753+0000","last_clean":"2024-09-11T01:19:00.440753+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:00.440753+0000","last_undegraded":"2024-09-11T01:19:00.440753+0000","last_fullsized":"2024-09-11T01:19:00.440753+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39306182,"num_objects":9598,"num_object_clones":0,"num_object_copies":19196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9598,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9024,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10655","reported_seq":10774,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.259078+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:00.259078+0000","last_peered":"2024-09-11T01:19:00.259078+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:00.259078+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461464,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8716,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3374},{"shards":"2","objects":5406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10815","reported_seq":10856,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.413841+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:00.413841+0000","last_peered":"2024-09-11T01:19:00.413841+0000","last_clean":"2024-09-11T01:19:00.413841+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:00.413841+0000","last_undegraded":"2024-09-11T01:19:00.413841+0000","last_fullsized":"2024-09-11T01:19:00.413841+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39104512,"num_objects":9547,"num_object_clones":0,"num_object_copies":19094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10678","reported_seq":10795,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:56.539774+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:18:56.539774+0000","last_peered":"2024-09-11T01:18:56.539774+0000","last_clean":"2024-09-11T01:18:56.539774+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:18:56.539774+0000","last_undegraded":"2024-09-11T01:18:56.539774+0000","last_fullsized":"2024-09-11T01:18:56.539774+0000","mapping_epoch":31,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317987,"num_objects":9600,"num_object_clones":0,"num_object_copies":19200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7476,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10642","reported_seq":10794,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.408950+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:18:56.408950+0000","last_peered":"2024-09-11T01:18:56.408950+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:18:56.408950+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39178723,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7130,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5462},{"shards":"0,2","objects":3578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'10954","reported_seq":11103,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.565857+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:18:56.565857+0000","last_peered":"2024-09-11T01:18:56.565857+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:18:56.565857+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40379334,"num_objects":9860,"num_object_clones":0,"num_object_copies":19720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9860,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7707,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5587},{"shards":"0,2","objects":3738}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10737","reported_seq":10852,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:56.600842+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:18:56.600842+0000","last_peered":"2024-09-11T01:18:56.600842+0000","last_clean":"2024-09-11T01:18:56.600842+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:18:56.600842+0000","last_undegraded":"2024-09-11T01:18:56.600842+0000","last_fullsized":"2024-09-11T01:18:56.600842+0000","mapping_epoch":31,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39677952,"num_objects":9687,"num_object_clones":0,"num_object_copies":19374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7690,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10710","reported_seq":19048,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:59.185552+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:59.185552+0000","last_peered":"2024-09-11T01:18:59.185552+0000","last_clean":"2024-09-11T01:18:59.185552+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:59.185552+0000","last_undegraded":"2024-09-11T01:18:59.185552+0000","last_fullsized":"2024-09-11T01:18:59.185552+0000","mapping_epoch":40,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39072227,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8577,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10870","reported_seq":11016,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.516659+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:18:56.516659+0000","last_peered":"2024-09-11T01:18:56.516659+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:18:56.516659+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39829504,"num_objects":9724,"num_object_clones":0,"num_object_copies":19448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4096,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7234,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5630},{"shards":"0,2","objects":3536}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10693","reported_seq":10803,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.350286+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:00.350286+0000","last_peered":"2024-09-11T01:19:00.350286+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:00.350286+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691299,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8462,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3406},{"shards":"2","objects":5430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10900","reported_seq":11019,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.199225+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:00.199225+0000","last_peered":"2024-09-11T01:19:00.199225+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:00.199225+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39575552,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8537,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3561},{"shards":"2","objects":5500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10661","reported_seq":13351,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:18:56.593232+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:18:56.593232+0000","last_peered":"2024-09-11T01:18:56.593232+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:18:56.593232+0000","last_undegraded":"2024-09-11T01:18:56.593232+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2882,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7351,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":5251072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6809},{"shards":"0,2","objects":2882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'10876","reported_seq":11034,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.610296+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:18:56.610296+0000","last_peered":"2024-09-11T01:18:56.610296+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:18:56.610296+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40017920,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5592},{"shards":"0,2","objects":3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10807","reported_seq":10956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.496198+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:18:56.496198+0000","last_peered":"2024-09-11T01:18:56.496198+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:18:56.496198+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40112128,"num_objects":9793,"num_object_clones":0,"num_object_copies":19586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7342,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5655},{"shards":"0,2","objects":3643}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10603","reported_seq":10739,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.503250+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:18:56.503250+0000","last_peered":"2024-09-11T01:18:56.503250+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:18:56.503250+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39056326,"num_objects":9537,"num_object_clones":0,"num_object_copies":19074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7260,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5386},{"shards":"0,2","objects":3626}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":630739702,"num_objects":153999,"num_object_clones":0,"num_object_copies":307998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41226,"num_objects_misplaced":2882,"num_objects_unfound":0,"num_objects_dirty":153999,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":131242,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5525,"num_bytes_recovered":22143459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1306660864,"data_stored":1306500974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709428,"kb_used_data":438940,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93662412,"statfs":{"total":96636764160,"available":95910309888,"internally_reserved":0,"allocated":449474560,"data_stored":449203183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789997,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596992,"kb_used_data":308464,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93774848,"statfs":{"total":96636764160,"available":96025444352,"internally_reserved":0,"allocated":315867136,"data_stored":315548326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881780,"kb_used_data":530848,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93490060,"statfs":{"total":96636764160,"available":95733821440,"internally_reserved":0,"allocated":543588352,"data_stored":543316970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,1,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":542662656,"data_stored":542626526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315396096,"data_stored":315312538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448602112,"data_stored":448561910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:02.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:02.857 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:02.857 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:02.887 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-11T01:19:02.504504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":625242889,"num_objects":152659,"num_object_clones":0,"num_object_copies":305318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":152659,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":132586,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5670,"num_bytes_recovered":22626806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2169428,"kb_used_data":1271960,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280946092,"statfs":{"total":289910292480,"available":287688798208,"internally_reserved":0,"allocated":1302487040,"data_stored":1301625471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,1,8,1,1,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22167552,"num_objects":-5412,"num_object_clones":0,"num_object_copies":-10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-86,"num_objects_misplaced":-354,"num_objects_unfound":0,"num_objects_dirty":-5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5412,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005233"},"pg_stats":[{"pgid":"2.f","version":"41'10788","reported_seq":10897,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.533163+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:01.533163+0000","last_peered":"2024-09-11T01:19:01.533163+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:01.533163+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10788,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10926","reported_seq":11002,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.440753+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:00.440753+0000","last_peered":"2024-09-11T01:19:00.440753+0000","last_clean":"2024-09-11T01:19:00.440753+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:00.440753+0000","last_undegraded":"2024-09-11T01:19:00.440753+0000","last_fullsized":"2024-09-11T01:19:00.440753+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39306182,"num_objects":9598,"num_object_clones":0,"num_object_copies":19196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9598,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9024,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10655","reported_seq":10774,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.259078+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:00.259078+0000","last_peered":"2024-09-11T01:19:00.259078+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:00.259078+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461464,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8716,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3374},{"shards":"2","objects":5406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10815","reported_seq":10856,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.413841+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:00.413841+0000","last_peered":"2024-09-11T01:19:00.413841+0000","last_clean":"2024-09-11T01:19:00.413841+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:00.413841+0000","last_undegraded":"2024-09-11T01:19:00.413841+0000","last_fullsized":"2024-09-11T01:19:00.413841+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39104512,"num_objects":9547,"num_object_clones":0,"num_object_copies":19094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10815","reported_seq":10932,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.603143+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:01.603143+0000","last_peered":"2024-09-11T01:19:01.603143+0000","last_clean":"2024-09-11T01:19:01.603143+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:01.603143+0000","last_undegraded":"2024-09-11T01:19:01.603143+0000","last_fullsized":"2024-09-11T01:19:01.603143+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756835,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7613,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10784","reported_seq":10948,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.643425+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:01.643425+0000","last_peered":"2024-09-11T01:19:01.643425+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:01.643425+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7272,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11108","reported_seq":11263,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.645706+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:01.645706+0000","last_peered":"2024-09-11T01:19:01.645706+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:01.645706+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7861,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5436},{"shards":"0,2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10855","reported_seq":10970,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.621892+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:01.621892+0000","last_peered":"2024-09-11T01:19:01.621892+0000","last_clean":"2024-09-11T01:19:01.621892+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:01.621892+0000","last_undegraded":"2024-09-11T01:19:01.621892+0000","last_fullsized":"2024-09-11T01:19:01.621892+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7808,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10710","reported_seq":19048,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:59.185552+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:59.185552+0000","last_peered":"2024-09-11T01:18:59.185552+0000","last_clean":"2024-09-11T01:18:59.185552+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:59.185552+0000","last_undegraded":"2024-09-11T01:18:59.185552+0000","last_fullsized":"2024-09-11T01:18:59.185552+0000","mapping_epoch":40,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39072227,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8577,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10988","reported_seq":11142,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.598580+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:01.598580+0000","last_peered":"2024-09-11T01:19:01.598580+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:01.598580+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7352,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10693","reported_seq":10803,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.350286+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:00.350286+0000","last_peered":"2024-09-11T01:19:00.350286+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:00.350286+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691299,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8462,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3406},{"shards":"2","objects":5430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10900","reported_seq":11019,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.199225+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:00.199225+0000","last_peered":"2024-09-11T01:19:00.199225+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:00.199225+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39575552,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8537,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3561},{"shards":"2","objects":5500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10791","reported_seq":13717,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:01.640122+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:01.640122+0000","last_peered":"2024-09-11T01:19:01.640122+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:01.640122+0000","last_undegraded":"2024-09-11T01:19:01.640122+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39161856,"num_objects":9561,"num_object_clones":0,"num_object_copies":19122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":9561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1400,"num_bytes_recovered":5734400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6797},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11022","reported_seq":11182,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.562570+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:01.562570+0000","last_peered":"2024-09-11T01:19:01.562570+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:01.562570+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10941","reported_seq":11092,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.546763+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:01.546763+0000","last_peered":"2024-09-11T01:19:01.546763+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:01.546763+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7476,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10730","reported_seq":10874,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.611567+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:01.611567+0000","last_peered":"2024-09-11T01:19:01.611567+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:01.611567+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38536134,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7387,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5263},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":625242870,"num_objects":152657,"num_object_clones":0,"num_object_copies":305314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":152657,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":132584,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5669,"num_bytes_recovered":22626787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300049920,"data_stored":1299890030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709428,"kb_used_data":438940,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93662412,"statfs":{"total":96636764160,"available":95910309888,"internally_reserved":0,"allocated":449474560,"data_stored":449203183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789997,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596992,"kb_used_data":308464,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93774848,"statfs":{"total":96636764160,"available":96025444352,"internally_reserved":0,"allocated":315867136,"data_stored":315548326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863008,"kb_used_data":524556,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93508832,"statfs":{"total":96636764160,"available":95753043968,"internally_reserved":0,"allocated":537145344,"data_stored":536873962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,1,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536051712,"data_stored":536015582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315396096,"data_stored":315312538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448602112,"data_stored":448561910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:02.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:02.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10788", 'reported_seq': 10897, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.533163+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:01.533163+0000', 'last_peered': '2024-09-11T01:19:01.533163+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:01.533163+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38707200, 'num_objects': 9450, 'num_object_clones': 0, 'num_object_copies': 18900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10788, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5347}, {'shards': '0,2', 'objects': 3454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:02.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10655", 'reported_seq': 10774, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:00.259078+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:00.259078+0000', 'last_peered': '2024-09-11T01:19:00.259078+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:00.259078+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'600", 'ondisk_log_start': "32'600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38461464, 'num_objects': 9391, 'num_object_clones': 0, 'num_object_copies': 18782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9391, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8716, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3374}, {'shards': '2', 'objects': 5406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'10784", 'reported_seq': 10948, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.643425+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:01.643425+0000', 'last_peered': '2024-09-11T01:19:01.643425+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:01.643425+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38597091, 'num_objects': 9424, 'num_object_clones': 0, 'num_object_copies': 18848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4099, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9424, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7272, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5326}, {'shards': '0,2', 'objects': 3436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11108", 'reported_seq': 11263, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.645706+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:01.645706+0000', 'last_peered': '2024-09-11T01:19:01.645706+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:01.645706+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39748550, 'num_objects': 9706, 'num_object_clones': 0, 'num_object_copies': 19412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4270, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9706, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7861, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5436}, {'shards': '0,2', 'objects': 3584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10988", 'reported_seq': 11142, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.598580+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:01.598580+0000', 'last_peered': '2024-09-11T01:19:01.598580+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:01.598580+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39346176, 'num_objects': 9606, 'num_object_clones': 0, 'num_object_copies': 19212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7352, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5516}, {'shards': '0,2', 'objects': 3418}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10693", 'reported_seq': 10803, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:00.350286+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:00.350286+0000', 'last_peered': '2024-09-11T01:19:00.350286+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:00.350286+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'600", 'ondisk_log_start': "32'600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38691299, 'num_objects': 9447, 'num_object_clones': 0, 'num_object_copies': 18894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4018, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9447, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8462, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3406}, {'shards': '2', 'objects': 5430}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:02.889 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'10900", 'reported_seq': 11019, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:00.199225+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:00.199225+0000', 'last_peered': '2024-09-11T01:19:00.199225+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:00.199225+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'800", 'ondisk_log_start': "32'800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39575552, 'num_objects': 9662, 'num_object_clones': 0, 'num_object_copies': 19324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8537, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3561}, {'shards': '2', 'objects': 5500}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10791", 'reported_seq': 13717, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:01.640122+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:01.640122+0000', 'last_peered': '2024-09-11T01:19:01.640122+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:01.640122+0000', 'last_undegraded': '2024-09-11T01:19:01.640122+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39161856, 'num_objects': 9561, 'num_object_clones': 0, 'num_object_copies': 19122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2764, 'num_objects_unfound': 0, 'num_objects_dirty': 9561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7481, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1400, 'num_bytes_recovered': 5734400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6797}, {'shards': '0,2', 'objects': 2764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11022", 'reported_seq': 11182, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.562570+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:01.562570+0000', 'last_peered': '2024-09-11T01:19:01.562570+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:01.562570+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39419904, 'num_objects': 9624, 'num_object_clones': 0, 'num_object_copies': 19248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4180, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7415, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5447}, {'shards': '0,2', 'objects': 3497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10941", 'reported_seq': 11092, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.546763+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:01.546763+0000', 'last_peered': '2024-09-11T01:19:01.546763+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:01.546763+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39563264, 'num_objects': 9659, 'num_object_clones': 0, 'num_object_copies': 19318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7476, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5522}, {'shards': '0,2', 'objects': 3509}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:02.890 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10730", 'reported_seq': 10874, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.611567+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:01.611567+0000', 'last_peered': '2024-09-11T01:19:01.611567+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:01.611567+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38536134, 'num_objects': 9410, 'num_object_clones': 0, 'num_object_copies': 18820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9410, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7387, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5263}, {'shards': '0,2', 'objects': 3499}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:02.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:03.607 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:03.607 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41200/305318 objects degraded (13.494%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":344,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152659,"data_bytes":625242889,"bytes_used":2221494272,"bytes_avail":287688798208,"bytes_total":289910292480,"degraded_objects":41200,"degraded_total":305318,"degraded_ratio":0.13494127434347139,"misplaced_objects":2764,"misplaced_total":305318,"misplaced_ratio":0.0090528563661493918,"recovering_objects_per_sec":36,"recovering_bytes_per_sec":120779,"recovering_keys_per_sec":0,"num_objects_recovered":440,"num_bytes_recovered":1449984,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":450},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:03.626 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:03.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:04.187 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:04.187 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:04.212 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-11T01:19:02.504504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":625242889,"num_objects":152659,"num_object_clones":0,"num_object_copies":305318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":152659,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":132586,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5670,"num_bytes_recovered":22626806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2169428,"kb_used_data":1271960,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280946092,"statfs":{"total":289910292480,"available":287688798208,"internally_reserved":0,"allocated":1302487040,"data_stored":1301625471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,1,8,1,1,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22167552,"num_objects":-5412,"num_object_clones":0,"num_object_copies":-10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-86,"num_objects_misplaced":-354,"num_objects_unfound":0,"num_objects_dirty":-5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5412,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1449984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005233"},"pg_stats":[{"pgid":"2.f","version":"41'10788","reported_seq":10897,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.533163+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:01.533163+0000","last_peered":"2024-09-11T01:19:01.533163+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:01.533163+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10788,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'10926","reported_seq":11002,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.440753+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:00.440753+0000","last_peered":"2024-09-11T01:19:00.440753+0000","last_clean":"2024-09-11T01:19:00.440753+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:00.440753+0000","last_undegraded":"2024-09-11T01:19:00.440753+0000","last_fullsized":"2024-09-11T01:19:00.440753+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39306182,"num_objects":9598,"num_object_clones":0,"num_object_copies":19196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9598,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9024,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10655","reported_seq":10774,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.259078+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:00.259078+0000","last_peered":"2024-09-11T01:19:00.259078+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:00.259078+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461464,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8716,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3374},{"shards":"2","objects":5406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10815","reported_seq":10856,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.413841+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:00.413841+0000","last_peered":"2024-09-11T01:19:00.413841+0000","last_clean":"2024-09-11T01:19:00.413841+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:00.413841+0000","last_undegraded":"2024-09-11T01:19:00.413841+0000","last_fullsized":"2024-09-11T01:19:00.413841+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39104512,"num_objects":9547,"num_object_clones":0,"num_object_copies":19094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10815,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10815","reported_seq":10932,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.603143+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:01.603143+0000","last_peered":"2024-09-11T01:19:01.603143+0000","last_clean":"2024-09-11T01:19:01.603143+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:01.603143+0000","last_undegraded":"2024-09-11T01:19:01.603143+0000","last_fullsized":"2024-09-11T01:19:01.603143+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756835,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7613,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10784","reported_seq":10948,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.643425+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:01.643425+0000","last_peered":"2024-09-11T01:19:01.643425+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:01.643425+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7272,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11108","reported_seq":11263,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.645706+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:01.645706+0000","last_peered":"2024-09-11T01:19:01.645706+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:01.645706+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7861,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5436},{"shards":"0,2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10855","reported_seq":10970,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.621892+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:01.621892+0000","last_peered":"2024-09-11T01:19:01.621892+0000","last_clean":"2024-09-11T01:19:01.621892+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:01.621892+0000","last_undegraded":"2024-09-11T01:19:01.621892+0000","last_fullsized":"2024-09-11T01:19:01.621892+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7808,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10710","reported_seq":19048,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:59.185552+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:18:59.185552+0000","last_peered":"2024-09-11T01:18:59.185552+0000","last_clean":"2024-09-11T01:18:59.185552+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:18:59.185552+0000","last_undegraded":"2024-09-11T01:18:59.185552+0000","last_fullsized":"2024-09-11T01:18:59.185552+0000","mapping_epoch":40,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39072227,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8577,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10988","reported_seq":11142,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.598580+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:01.598580+0000","last_peered":"2024-09-11T01:19:01.598580+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:01.598580+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7352,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10693","reported_seq":10803,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.350286+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:00.350286+0000","last_peered":"2024-09-11T01:19:00.350286+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:00.350286+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'600","ondisk_log_start":"32'600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38691299,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8462,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3406},{"shards":"2","objects":5430}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'10900","reported_seq":11019,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:00.199225+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:00.199225+0000","last_peered":"2024-09-11T01:19:00.199225+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:00.199225+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39575552,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8537,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3561},{"shards":"2","objects":5500}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10791","reported_seq":13717,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:01.640122+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:01.640122+0000","last_peered":"2024-09-11T01:19:01.640122+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:01.640122+0000","last_undegraded":"2024-09-11T01:19:01.640122+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39161856,"num_objects":9561,"num_object_clones":0,"num_object_copies":19122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":9561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1400,"num_bytes_recovered":5734400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6797},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11022","reported_seq":11182,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.562570+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:01.562570+0000","last_peered":"2024-09-11T01:19:01.562570+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:01.562570+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10941","reported_seq":11092,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.546763+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:01.546763+0000","last_peered":"2024-09-11T01:19:01.546763+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:01.546763+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7476,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10730","reported_seq":10874,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.611567+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:01.611567+0000","last_peered":"2024-09-11T01:19:01.611567+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:01.611567+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38536134,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7387,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5263},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":625242870,"num_objects":152657,"num_object_clones":0,"num_object_copies":305314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41200,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":152657,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":132584,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5669,"num_bytes_recovered":22626787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300049920,"data_stored":1299890030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709428,"kb_used_data":438940,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93662412,"statfs":{"total":96636764160,"available":95910309888,"internally_reserved":0,"allocated":449474560,"data_stored":449203183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":276950702},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789997,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":596992,"kb_used_data":308464,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93774848,"statfs":{"total":96636764160,"available":96025444352,"internally_reserved":0,"allocated":315867136,"data_stored":315548326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863008,"kb_used_data":524556,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93508832,"statfs":{"total":96636764160,"available":95753043968,"internally_reserved":0,"allocated":537145344,"data_stored":536873962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,1,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536051712,"data_stored":536015582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315396096,"data_stored":315312538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448602112,"data_stored":448561910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:07.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:07.771 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:07.771 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:07.794 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-11T01:19:06.505553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622064393,"num_objects":151883,"num_object_clones":0,"num_object_copies":303766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41198,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":151883,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":133362,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5672,"num_bytes_recovered":22626806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2160540,"kb_used_data":1267236,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":280954980,"statfs":{"total":289910292480,"available":287697899520,"internally_reserved":0,"allocated":1297649664,"data_stored":1296788095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":914672647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,4,9,3,1,0,0,0,1,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19460096,"num_objects":-4751,"num_object_clones":0,"num_object_copies":-9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"2.f","version":"41'10788","reported_seq":10897,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.533163+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:01.533163+0000","last_peered":"2024-09-11T01:19:01.533163+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:01.533163+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10788,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11050","reported_seq":11126,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.376803+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:05.376803+0000","last_peered":"2024-09-11T01:19:05.376803+0000","last_clean":"2024-09-11T01:19:05.376803+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:05.376803+0000","last_undegraded":"2024-09-11T01:19:05.376803+0000","last_fullsized":"2024-09-11T01:19:05.376803+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38798278,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9148,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10784","reported_seq":10905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.174054+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:05.174054+0000","last_peered":"2024-09-11T01:19:05.174054+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:05.174054+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933080,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8845,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3245},{"shards":"2","objects":5278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10942","reported_seq":10983,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.387418+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:05.387418+0000","last_peered":"2024-09-11T01:19:05.387418+0000","last_clean":"2024-09-11T01:19:05.387418+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:05.387418+0000","last_undegraded":"2024-09-11T01:19:05.387418+0000","last_fullsized":"2024-09-11T01:19:05.387418+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10942,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10815","reported_seq":10932,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.603143+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:01.603143+0000","last_peered":"2024-09-11T01:19:01.603143+0000","last_clean":"2024-09-11T01:19:01.603143+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:01.603143+0000","last_undegraded":"2024-09-11T01:19:01.603143+0000","last_fullsized":"2024-09-11T01:19:01.603143+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756835,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7613,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10784","reported_seq":10948,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.643425+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:01.643425+0000","last_peered":"2024-09-11T01:19:01.643425+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:01.643425+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7272,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11108","reported_seq":11263,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.645706+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:01.645706+0000","last_peered":"2024-09-11T01:19:01.645706+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:01.645706+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7861,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5436},{"shards":"0,2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10855","reported_seq":10970,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.621892+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:01.621892+0000","last_peered":"2024-09-11T01:19:01.621892+0000","last_clean":"2024-09-11T01:19:01.621892+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:01.621892+0000","last_undegraded":"2024-09-11T01:19:01.621892+0000","last_fullsized":"2024-09-11T01:19:01.621892+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7808,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10845","reported_seq":19183,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.206046+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:04.206046+0000","last_peered":"2024-09-11T01:19:04.206046+0000","last_clean":"2024-09-11T01:19:04.206046+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:04.206046+0000","last_undegraded":"2024-09-11T01:19:04.206046+0000","last_fullsized":"2024-09-11T01:19:04.206046+0000","mapping_epoch":40,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38519267,"num_objects":9405,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8712,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10988","reported_seq":11142,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.598580+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:01.598580+0000","last_peered":"2024-09-11T01:19:01.598580+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:01.598580+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7352,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10818","reported_seq":10930,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.414666+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:05.414666+0000","last_peered":"2024-09-11T01:19:05.414666+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:05.414666+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8587,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3281},{"shards":"2","objects":5306}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11036","reported_seq":11155,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.402769+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:05.402769+0000","last_peered":"2024-09-11T01:19:05.402769+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:05.402769+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8673,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3425},{"shards":"2","objects":5364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10791","reported_seq":13717,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:01.640122+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:01.640122+0000","last_peered":"2024-09-11T01:19:01.640122+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:01.640122+0000","last_undegraded":"2024-09-11T01:19:01.640122+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39161856,"num_objects":9561,"num_object_clones":0,"num_object_copies":19122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":9561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1400,"num_bytes_recovered":5734400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6797},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11022","reported_seq":11182,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.562570+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:01.562570+0000","last_peered":"2024-09-11T01:19:01.562570+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:01.562570+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10941","reported_seq":11092,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.546763+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:01.546763+0000","last_peered":"2024-09-11T01:19:01.546763+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:01.546763+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7476,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10730","reported_seq":10874,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.611567+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:01.611567+0000","last_peered":"2024-09-11T01:19:01.611567+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:01.611567+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38536134,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7387,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5263},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622064374,"num_objects":151881,"num_object_clones":0,"num_object_copies":303762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41198,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":151881,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":133360,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5671,"num_bytes_recovered":22626787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1295208448,"data_stored":1295048558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709448,"kb_used_data":434804,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93662392,"statfs":{"total":96636764160,"available":95910289408,"internally_reserved":0,"allocated":445239296,"data_stored":444967919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281210542},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,1,1,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588084,"kb_used_data":307876,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93783756,"statfs":{"total":96636764160,"available":96034566144,"internally_reserved":0,"allocated":315265024,"data_stored":314946214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863008,"kb_used_data":524556,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93508832,"statfs":{"total":96636764160,"available":95753043968,"internally_reserved":0,"allocated":537145344,"data_stored":536873962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,1,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536051712,"data_stored":536015582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314826752,"data_stored":314743194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444329984,"data_stored":444289782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:07.795 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:08.363 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:08.364 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:08.389 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-11T01:19:06.505553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":622064393,"num_objects":151883,"num_object_clones":0,"num_object_copies":303766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41198,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":151883,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":133362,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5672,"num_bytes_recovered":22626806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2160540,"kb_used_data":1267236,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":280954980,"statfs":{"total":289910292480,"available":287697899520,"internally_reserved":0,"allocated":1297649664,"data_stored":1296788095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":914672647},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,4,9,3,1,0,0,0,1,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19460096,"num_objects":-4751,"num_object_clones":0,"num_object_copies":-9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-79,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":-4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002991"},"pg_stats":[{"pgid":"2.f","version":"41'10788","reported_seq":10897,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.533163+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:01.533163+0000","last_peered":"2024-09-11T01:19:01.533163+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:01.533163+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10788,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5347},{"shards":"0,2","objects":3454}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11050","reported_seq":11126,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.376803+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:05.376803+0000","last_peered":"2024-09-11T01:19:05.376803+0000","last_clean":"2024-09-11T01:19:05.376803+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:05.376803+0000","last_undegraded":"2024-09-11T01:19:05.376803+0000","last_fullsized":"2024-09-11T01:19:05.376803+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38798278,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9148,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10784","reported_seq":10905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.174054+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:05.174054+0000","last_peered":"2024-09-11T01:19:05.174054+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:05.174054+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933080,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8845,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3245},{"shards":"2","objects":5278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10942","reported_seq":10983,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.387418+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:05.387418+0000","last_peered":"2024-09-11T01:19:05.387418+0000","last_clean":"2024-09-11T01:19:05.387418+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:05.387418+0000","last_undegraded":"2024-09-11T01:19:05.387418+0000","last_fullsized":"2024-09-11T01:19:05.387418+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10942,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10815","reported_seq":10932,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.603143+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:01.603143+0000","last_peered":"2024-09-11T01:19:01.603143+0000","last_clean":"2024-09-11T01:19:01.603143+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:01.603143+0000","last_undegraded":"2024-09-11T01:19:01.603143+0000","last_fullsized":"2024-09-11T01:19:01.603143+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38756835,"num_objects":9463,"num_object_clones":0,"num_object_copies":18926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9463,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7613,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10784","reported_seq":10948,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.643425+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:01.643425+0000","last_peered":"2024-09-11T01:19:01.643425+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:01.643425+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7272,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11108","reported_seq":11263,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.645706+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:01.645706+0000","last_peered":"2024-09-11T01:19:01.645706+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:01.645706+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39748550,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7861,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5436},{"shards":"0,2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10855","reported_seq":10970,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.621892+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:01.621892+0000","last_peered":"2024-09-11T01:19:01.621892+0000","last_clean":"2024-09-11T01:19:01.621892+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:01.621892+0000","last_undegraded":"2024-09-11T01:19:01.621892+0000","last_fullsized":"2024-09-11T01:19:01.621892+0000","mapping_epoch":31,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39194624,"num_objects":9569,"num_object_clones":0,"num_object_copies":19138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7808,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10845","reported_seq":19183,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.206046+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:04.206046+0000","last_peered":"2024-09-11T01:19:04.206046+0000","last_clean":"2024-09-11T01:19:04.206046+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:04.206046+0000","last_undegraded":"2024-09-11T01:19:04.206046+0000","last_fullsized":"2024-09-11T01:19:04.206046+0000","mapping_epoch":40,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38519267,"num_objects":9405,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8712,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'10988","reported_seq":11142,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.598580+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:01.598580+0000","last_peered":"2024-09-11T01:19:01.598580+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:01.598580+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39346176,"num_objects":9606,"num_object_clones":0,"num_object_copies":19212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7352,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3418}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10818","reported_seq":10930,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.414666+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:05.414666+0000","last_peered":"2024-09-11T01:19:05.414666+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:05.414666+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8587,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3281},{"shards":"2","objects":5306}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11036","reported_seq":11155,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.402769+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:05.402769+0000","last_peered":"2024-09-11T01:19:05.402769+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:05.402769+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8673,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3425},{"shards":"2","objects":5364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10791","reported_seq":13717,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:01.640122+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:01.640122+0000","last_peered":"2024-09-11T01:19:01.640122+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:01.640122+0000","last_undegraded":"2024-09-11T01:19:01.640122+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39161856,"num_objects":9561,"num_object_clones":0,"num_object_copies":19122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":9561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7481,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1400,"num_bytes_recovered":5734400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6797},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11022","reported_seq":11182,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.562570+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:01.562570+0000","last_peered":"2024-09-11T01:19:01.562570+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:01.562570+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39419904,"num_objects":9624,"num_object_clones":0,"num_object_copies":19248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7415,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5447},{"shards":"0,2","objects":3497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'10941","reported_seq":11092,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.546763+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:01.546763+0000","last_peered":"2024-09-11T01:19:01.546763+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:01.546763+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7476,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5522},{"shards":"0,2","objects":3509}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10730","reported_seq":10874,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.611567+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:01.611567+0000","last_peered":"2024-09-11T01:19:01.611567+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:01.611567+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38536134,"num_objects":9410,"num_object_clones":0,"num_object_copies":18820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7387,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5263},{"shards":"0,2","objects":3499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":622064374,"num_objects":151881,"num_object_clones":0,"num_object_copies":303762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41198,"num_objects_misplaced":2764,"num_objects_unfound":0,"num_objects_dirty":151881,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":133360,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5671,"num_bytes_recovered":22626787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1295208448,"data_stored":1295048558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709448,"kb_used_data":434804,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93662392,"statfs":{"total":96636764160,"available":95910289408,"internally_reserved":0,"allocated":445239296,"data_stored":444967919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281210542},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,1,1,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588084,"kb_used_data":307876,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93783756,"statfs":{"total":96636764160,"available":96034566144,"internally_reserved":0,"allocated":315265024,"data_stored":314946214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863008,"kb_used_data":524556,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93508832,"statfs":{"total":96636764160,"available":95753043968,"internally_reserved":0,"allocated":537145344,"data_stored":536873962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,4,1,0,0,0,0,0,8],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":536051712,"data_stored":536015582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314826752,"data_stored":314743194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444329984,"data_stored":444289782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:08.390 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:08.390 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'10788", 'reported_seq': 10897, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.533163+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:01.533163+0000', 'last_peered': '2024-09-11T01:19:01.533163+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:01.533163+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38707200, 'num_objects': 9450, 'num_object_clones': 0, 'num_object_copies': 18900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10788, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5347}, {'shards': '0,2', 'objects': 3454}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.390 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10784", 'reported_seq': 10905, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:05.174054+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:05.174054+0000', 'last_peered': '2024-09-11T01:19:05.174054+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:05.174054+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37933080, 'num_objects': 9262, 'num_object_clones': 0, 'num_object_copies': 18524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9262, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8845, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3245}, {'shards': '2', 'objects': 5278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'10784", 'reported_seq': 10948, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.643425+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:01.643425+0000', 'last_peered': '2024-09-11T01:19:01.643425+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:01.643425+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38597091, 'num_objects': 9424, 'num_object_clones': 0, 'num_object_copies': 18848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4099, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9424, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7272, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5326}, {'shards': '0,2', 'objects': 3436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11108", 'reported_seq': 11263, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.645706+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:01.645706+0000', 'last_peered': '2024-09-11T01:19:01.645706+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:01.645706+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39748550, 'num_objects': 9706, 'num_object_clones': 0, 'num_object_copies': 19412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4270, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9706, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 7861, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 13, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5436}, {'shards': '0,2', 'objects': 3584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'10988", 'reported_seq': 11142, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.598580+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:01.598580+0000', 'last_peered': '2024-09-11T01:19:01.598580+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:01.598580+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39346176, 'num_objects': 9606, 'num_object_clones': 0, 'num_object_copies': 19212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9606, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7352, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5516}, {'shards': '0,2', 'objects': 3418}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10818", 'reported_seq': 10930, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:05.414666+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:05.414666+0000', 'last_peered': '2024-09-11T01:19:05.414666+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:05.414666+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'800", 'ondisk_log_start': "32'800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38179299, 'num_objects': 9322, 'num_object_clones': 0, 'num_object_copies': 18644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9322, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8587, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3281}, {'shards': '2', 'objects': 5306}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:08.391 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11036", 'reported_seq': 11155, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:05.402769+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:05.402769+0000', 'last_peered': '2024-09-11T01:19:05.402769+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:05.402769+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39018496, 'num_objects': 9526, 'num_object_clones': 0, 'num_object_copies': 19052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8673, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3425}, {'shards': '2', 'objects': 5364}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'10791", 'reported_seq': 13717, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:01.640122+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:01.640122+0000', 'last_peered': '2024-09-11T01:19:01.640122+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:01.640122+0000', 'last_undegraded': '2024-09-11T01:19:01.640122+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39161856, 'num_objects': 9561, 'num_object_clones': 0, 'num_object_copies': 19122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2764, 'num_objects_unfound': 0, 'num_objects_dirty': 9561, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7481, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1400, 'num_bytes_recovered': 5734400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6797}, {'shards': '0,2', 'objects': 2764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11022", 'reported_seq': 11182, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.562570+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:01.562570+0000', 'last_peered': '2024-09-11T01:19:01.562570+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:01.562570+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39419904, 'num_objects': 9624, 'num_object_clones': 0, 'num_object_copies': 19248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4180, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7415, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5447}, {'shards': '0,2', 'objects': 3497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'10941", 'reported_seq': 11092, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.546763+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:01.546763+0000', 'last_peered': '2024-09-11T01:19:01.546763+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:01.546763+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39563264, 'num_objects': 9659, 'num_object_clones': 0, 'num_object_copies': 19318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7476, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5522}, {'shards': '0,2', 'objects': 3509}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:08.392 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10730", 'reported_seq': 10874, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.611567+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:01.611567+0000', 'last_peered': '2024-09-11T01:19:01.611567+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:01.611567+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'700", 'ondisk_log_start': "32'700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38536134, 'num_objects': 9410, 'num_object_clones': 0, 'num_object_copies': 18820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9410, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7387, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 7, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5263}, {'shards': '0,2', 'objects': 3499}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:08.393 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:09.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:09.100 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41177/301128 objects degraded (13.674%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":349,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150564,"data_bytes":616661769,"bytes_used":2210242560,"bytes_avail":287700049920,"bytes_total":289910292480,"degraded_objects":41177,"degraded_total":301128,"degraded_ratio":0.13674251481097738,"misplaced_objects":2637,"misplaced_total":301128,"misplaced_ratio":0.0087570734040009566,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":128306,"recovering_keys_per_sec":0,"num_objects_recovered":462,"num_bytes_recovered":1540096,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":451},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:09.125 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:09.125 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:09.707 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:09.707 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:09.734 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-11T01:19:08.506242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":616661769,"num_objects":150564,"num_object_clones":0,"num_object_copies":301128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41177,"num_objects_misplaced":2637,"num_objects_unfound":0,"num_objects_dirty":150564,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":134681,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5820,"num_bytes_recovered":23146998,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160719,"ondisk_log_size":160719,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2158440,"kb_used_data":1260980,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280957080,"statfs":{"total":289910292480,"available":287700049920,"internally_reserved":0,"allocated":1291243520,"data_stored":1290381951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,5,8,3,1,0,8,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22208512,"num_objects":-5422,"num_object_clones":0,"num_object_copies":-10844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-86,"num_objects_misplaced":-376,"num_objects_unfound":0,"num_objects_dirty":-5422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5422,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":1540096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003213"},"pg_stats":[{"pgid":"2.f","version":"41'10923","reported_seq":11040,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.685359+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:06.685359+0000","last_peered":"2024-09-11T01:19:06.685359+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:06.685359+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38154240,"num_objects":9315,"num_object_clones":0,"num_object_copies":18630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10923,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5216},{"shards":"0,2","objects":3319}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11050","reported_seq":11126,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.376803+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:05.376803+0000","last_peered":"2024-09-11T01:19:05.376803+0000","last_clean":"2024-09-11T01:19:05.376803+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:05.376803+0000","last_undegraded":"2024-09-11T01:19:05.376803+0000","last_fullsized":"2024-09-11T01:19:05.376803+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38798278,"num_objects":9474,"num_object_clones":0,"num_object_copies":18948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9474,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9148,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10784","reported_seq":10905,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.174054+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:05.174054+0000","last_peered":"2024-09-11T01:19:05.174054+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:05.174054+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'700","ondisk_log_start":"32'700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933080,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8845,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3245},{"shards":"2","objects":5278}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'10942","reported_seq":10983,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.387418+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:05.387418+0000","last_peered":"2024-09-11T01:19:05.387418+0000","last_clean":"2024-09-11T01:19:05.387418+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:05.387418+0000","last_undegraded":"2024-09-11T01:19:05.387418+0000","last_fullsized":"2024-09-11T01:19:05.387418+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10942,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'10950","reported_seq":11067,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:06.632681+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:06.632681+0000","last_peered":"2024-09-11T01:19:06.632681+0000","last_clean":"2024-09-11T01:19:06.632681+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:06.632681+0000","last_undegraded":"2024-09-11T01:19:06.632681+0000","last_fullsized":"2024-09-11T01:19:06.632681+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38203875,"num_objects":9328,"num_object_clones":0,"num_object_copies":18656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7748,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'10902","reported_seq":11066,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.635911+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:06.635911+0000","last_peered":"2024-09-11T01:19:06.635911+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:06.635911+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38113763,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7390,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5208},{"shards":"0,2","objects":3318}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11236","reported_seq":11401,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.704932+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:06.704932+0000","last_peered":"2024-09-11T01:19:06.704932+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:06.704932+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39224262,"num_objects":9578,"num_object_clones":0,"num_object_copies":19156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9578,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7989,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5313},{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'10991","reported_seq":11106,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:06.620247+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:06.620247+0000","last_peered":"2024-09-11T01:19:06.620247+0000","last_clean":"2024-09-11T01:19:06.620247+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:06.620247+0000","last_undegraded":"2024-09-11T01:19:06.620247+0000","last_fullsized":"2024-09-11T01:19:06.620247+0000","mapping_epoch":31,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38637568,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7944,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10845","reported_seq":19183,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.206046+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:04.206046+0000","last_peered":"2024-09-11T01:19:04.206046+0000","last_clean":"2024-09-11T01:19:04.206046+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:04.206046+0000","last_undegraded":"2024-09-11T01:19:04.206046+0000","last_fullsized":"2024-09-11T01:19:04.206046+0000","mapping_epoch":40,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38519267,"num_objects":9405,"num_object_clones":0,"num_object_copies":18810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8712,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11113","reported_seq":11269,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.703168+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:06.703168+0000","last_peered":"2024-09-11T01:19:06.703168+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:06.703168+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38834176,"num_objects":9481,"num_object_clones":0,"num_object_copies":18962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7477,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5393},{"shards":"0,2","objects":3293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10818","reported_seq":10930,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.414666+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:05.414666+0000","last_peered":"2024-09-11T01:19:05.414666+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:05.414666+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38179299,"num_objects":9322,"num_object_clones":0,"num_object_copies":18644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8587,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3281},{"shards":"2","objects":5306}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11036","reported_seq":11155,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:05.402769+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:05.402769+0000","last_peered":"2024-09-11T01:19:05.402769+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:05.402769+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018496,"num_objects":9526,"num_object_clones":0,"num_object_copies":19052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8673,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3425},{"shards":"2","objects":5364}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'10930","reported_seq":14110,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:06.722564+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:06.722564+0000","last_peered":"2024-09-11T01:19:06.722564+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:06.722564+0000","last_undegraded":"2024-09-11T01:19:06.722564+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38592512,"num_objects":9422,"num_object_clones":0,"num_object_copies":18844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2637,"num_objects_unfound":0,"num_objects_dirty":9422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7620,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1527,"num_bytes_recovered":6254592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6785},{"shards":"0,2","objects":2637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11154","reported_seq":11322,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.701313+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:06.701313+0000","last_peered":"2024-09-11T01:19:06.701313+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:06.701313+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38879232,"num_objects":9492,"num_object_clones":0,"num_object_copies":18984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5319},{"shards":"0,2","objects":3365}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11081","reported_seq":11240,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.684959+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:06.684959+0000","last_peered":"2024-09-11T01:19:06.684959+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:06.684959+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38989824,"num_objects":9519,"num_object_clones":0,"num_object_copies":19038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7616,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5386},{"shards":"0,2","objects":3369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10861","reported_seq":11009,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.676555+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:06.676555+0000","last_peered":"2024-09-11T01:19:06.676555+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:06.676555+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'800","ondisk_log_start":"32'800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37999558,"num_objects":9279,"num_object_clones":0,"num_object_copies":18558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9279,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7518,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":616661750,"num_objects":150562,"num_object_clones":0,"num_object_copies":301124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41177,"num_objects_misplaced":2637,"num_objects_unfound":0,"num_objects_dirty":150562,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":134679,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5819,"num_bytes_recovered":23146979,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1288810496,"data_stored":1288650606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160716,"ondisk_log_size":160716,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709448,"kb_used_data":434804,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93662392,"statfs":{"total":96636764160,"available":95910289408,"internally_reserved":0,"allocated":445239296,"data_stored":444967919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":281210542},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,1,1,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789998,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588084,"kb_used_data":307876,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93783756,"statfs":{"total":96636764160,"available":96034566144,"internally_reserved":0,"allocated":315265024,"data_stored":314946214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860908,"kb_used_data":518300,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93510932,"statfs":{"total":96636764160,"available":95755194368,"internally_reserved":0,"allocated":530739200,"data_stored":530467818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,3,1,0,0,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":529653760,"data_stored":529617630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314826752,"data_stored":314743194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":444329984,"data_stored":444289782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:12.737 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:13.326 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:13.326 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:13.358 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T01:19:12.507764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607859465,"num_objects":148415,"num_object_clones":0,"num_object_copies":296830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41169,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":148415,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":136830,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":23581174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160968,"ondisk_log_size":160968,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147320,"kb_used_data":1249844,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280968200,"statfs":{"total":289910292480,"available":287711436800,"internally_reserved":0,"allocated":1279840256,"data_stored":1278978687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,10,5,0,0,0,0,0,9],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22880256,"num_objects":-5586,"num_object_clones":0,"num_object_copies":-11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-351,"num_objects_unfound":0,"num_objects_dirty":-5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5586,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003764"},"pg_stats":[{"pgid":"2.f","version":"41'11053","reported_seq":11171,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.757904+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:11.757904+0000","last_peered":"2024-09-11T01:19:11.757904+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:11.757904+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11053,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5088},{"shards":"0,2","objects":3189}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11171","reported_seq":11247,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:10.435171+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:10.435171+0000","last_peered":"2024-09-11T01:19:10.435171+0000","last_clean":"2024-09-11T01:19:10.435171+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:10.435171+0000","last_undegraded":"2024-09-11T01:19:10.435171+0000","last_fullsized":"2024-09-11T01:19:10.435171+0000","mapping_epoch":31,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38302662,"num_objects":9353,"num_object_clones":0,"num_object_copies":18706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9353,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9269,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10905","reported_seq":11026,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.396569+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:10.396569+0000","last_peered":"2024-09-11T01:19:10.396569+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:10.396569+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37437464,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8966,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3124},{"shards":"2","objects":5157}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11072","reported_seq":11113,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:10.394968+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:10.394968+0000","last_peered":"2024-09-11T01:19:10.394968+0000","last_clean":"2024-09-11T01:19:10.394968+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:10.394968+0000","last_undegraded":"2024-09-11T01:19:10.394968+0000","last_fullsized":"2024-09-11T01:19:10.394968+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11072,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11088","reported_seq":11205,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.719253+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:11.719253+0000","last_peered":"2024-09-11T01:19:11.719253+0000","last_clean":"2024-09-11T01:19:11.719253+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:11.719253+0000","last_undegraded":"2024-09-11T01:19:11.719253+0000","last_fullsized":"2024-09-11T01:19:11.719253+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638627,"num_objects":9190,"num_object_clones":0,"num_object_copies":18380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7886,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11034","reported_seq":11202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.708913+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:11.708913+0000","last_peered":"2024-09-11T01:19:11.708913+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:11.708913+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37573091,"num_objects":9174,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7522,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":3186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11381","reported_seq":11546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.753776+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:11.753776+0000","last_peered":"2024-09-11T01:19:11.753776+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:11.753776+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38630342,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8134,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5168},{"shards":"0,2","objects":3311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11123","reported_seq":11238,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.715764+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:11.715764+0000","last_peered":"2024-09-11T01:19:11.715764+0000","last_clean":"2024-09-11T01:19:11.715764+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:11.715764+0000","last_undegraded":"2024-09-11T01:19:11.715764+0000","last_fullsized":"2024-09-11T01:19:11.715764+0000","mapping_epoch":31,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38096896,"num_objects":9301,"num_object_clones":0,"num_object_copies":18602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8076,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10994","reported_seq":19332,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.180566+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:09.180566+0000","last_peered":"2024-09-11T01:19:09.180566+0000","last_clean":"2024-09-11T01:19:09.180566+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:09.180566+0000","last_undegraded":"2024-09-11T01:19:09.180566+0000","last_fullsized":"2024-09-11T01:19:09.180566+0000","mapping_epoch":40,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37908963,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8861,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11254","reported_seq":11412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.652911+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:11.652911+0000","last_peered":"2024-09-11T01:19:11.652911+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:11.652911+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38256640,"num_objects":9340,"num_object_clones":0,"num_object_copies":18680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7618,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5252},{"shards":"0,2","objects":3152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10968","reported_seq":11080,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.422798+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:10.422798+0000","last_peered":"2024-09-11T01:19:10.422798+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:10.422798+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564899,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8737,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3131},{"shards":"2","objects":5156}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11159","reported_seq":11278,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.441474+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:10.441474+0000","last_peered":"2024-09-11T01:19:10.441474+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:10.441474+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8796,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3302},{"shards":"2","objects":5241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11057","reported_seq":14449,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:11.733787+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:11.733787+0000","last_peered":"2024-09-11T01:19:11.733787+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:11.733787+0000","last_undegraded":"2024-09-11T01:19:11.733787+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1633,"num_bytes_recovered":6688768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6764},{"shards":"0,2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11289","reported_seq":11459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.688883+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:11.688883+0000","last_peered":"2024-09-11T01:19:11.688883+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:11.688883+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5185},{"shards":"0,2","objects":3230}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11224","reported_seq":11385,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.733327+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:11.733327+0000","last_peered":"2024-09-11T01:19:11.733327+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:11.733327+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38404096,"num_objects":9376,"num_object_clones":0,"num_object_copies":18752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10993","reported_seq":11145,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.652744+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:11.652744+0000","last_peered":"2024-09-11T01:19:11.652744+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:11.652744+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37458886,"num_objects":9147,"num_object_clones":0,"num_object_copies":18294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7650,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5004},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607859446,"num_objects":148413,"num_object_clones":0,"num_object_copies":296826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41169,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":148413,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":136828,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5933,"num_bytes_recovered":23581155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1277243392,"data_stored":1277083502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692796,"kb_used_data":430632,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93679044,"statfs":{"total":96636764160,"available":95927341056,"internally_reserved":0,"allocated":440967168,"data_stored":440695791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,3,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591688,"kb_used_data":307320,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93780152,"statfs":{"total":96636764160,"available":96030875648,"internally_reserved":0,"allocated":314695680,"data_stored":314376870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":862836,"kb_used_data":511892,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93509004,"statfs":{"total":96636764160,"available":95753220096,"internally_reserved":0,"allocated":524177408,"data_stored":523906026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,2,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523055104,"data_stored":523018974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314163200,"data_stored":314079642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440025088,"data_stored":439984886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:13.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:13.947 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:13.947 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:13.970 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":211,"stamp":"2024-09-11T01:19:12.507764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607859465,"num_objects":148415,"num_object_clones":0,"num_object_copies":296830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41169,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":148415,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":136830,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":23581174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160968,"ondisk_log_size":160968,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147320,"kb_used_data":1249844,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280968200,"statfs":{"total":289910292480,"available":287711436800,"internally_reserved":0,"allocated":1279840256,"data_stored":1278978687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,10,5,0,0,0,0,0,9],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22880256,"num_objects":-5586,"num_object_clones":0,"num_object_copies":-11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-351,"num_objects_unfound":0,"num_objects_dirty":-5586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5586,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003764"},"pg_stats":[{"pgid":"2.f","version":"41'11053","reported_seq":11171,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.757904+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:11.757904+0000","last_peered":"2024-09-11T01:19:11.757904+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:11.757904+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11053,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5088},{"shards":"0,2","objects":3189}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11171","reported_seq":11247,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:10.435171+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:10.435171+0000","last_peered":"2024-09-11T01:19:10.435171+0000","last_clean":"2024-09-11T01:19:10.435171+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:10.435171+0000","last_undegraded":"2024-09-11T01:19:10.435171+0000","last_fullsized":"2024-09-11T01:19:10.435171+0000","mapping_epoch":31,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38302662,"num_objects":9353,"num_object_clones":0,"num_object_copies":18706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9353,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9269,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10905","reported_seq":11026,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.396569+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:10.396569+0000","last_peered":"2024-09-11T01:19:10.396569+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:10.396569+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37437464,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8966,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3124},{"shards":"2","objects":5157}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11072","reported_seq":11113,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:10.394968+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:10.394968+0000","last_peered":"2024-09-11T01:19:10.394968+0000","last_clean":"2024-09-11T01:19:10.394968+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:10.394968+0000","last_undegraded":"2024-09-11T01:19:10.394968+0000","last_fullsized":"2024-09-11T01:19:10.394968+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11072,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11088","reported_seq":11205,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.719253+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:11.719253+0000","last_peered":"2024-09-11T01:19:11.719253+0000","last_clean":"2024-09-11T01:19:11.719253+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:11.719253+0000","last_undegraded":"2024-09-11T01:19:11.719253+0000","last_fullsized":"2024-09-11T01:19:11.719253+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638627,"num_objects":9190,"num_object_clones":0,"num_object_copies":18380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7886,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11034","reported_seq":11202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.708913+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:11.708913+0000","last_peered":"2024-09-11T01:19:11.708913+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:11.708913+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37573091,"num_objects":9174,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7522,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":3186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11381","reported_seq":11546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.753776+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:11.753776+0000","last_peered":"2024-09-11T01:19:11.753776+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:11.753776+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38630342,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8134,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5168},{"shards":"0,2","objects":3311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11123","reported_seq":11238,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.715764+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:11.715764+0000","last_peered":"2024-09-11T01:19:11.715764+0000","last_clean":"2024-09-11T01:19:11.715764+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:11.715764+0000","last_undegraded":"2024-09-11T01:19:11.715764+0000","last_fullsized":"2024-09-11T01:19:11.715764+0000","mapping_epoch":31,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38096896,"num_objects":9301,"num_object_clones":0,"num_object_copies":18602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8076,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'10994","reported_seq":19332,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.180566+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:09.180566+0000","last_peered":"2024-09-11T01:19:09.180566+0000","last_clean":"2024-09-11T01:19:09.180566+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:09.180566+0000","last_undegraded":"2024-09-11T01:19:09.180566+0000","last_fullsized":"2024-09-11T01:19:09.180566+0000","mapping_epoch":40,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37908963,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8861,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11254","reported_seq":11412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.652911+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:11.652911+0000","last_peered":"2024-09-11T01:19:11.652911+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:11.652911+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38256640,"num_objects":9340,"num_object_clones":0,"num_object_copies":18680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7618,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5252},{"shards":"0,2","objects":3152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10968","reported_seq":11080,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.422798+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:10.422798+0000","last_peered":"2024-09-11T01:19:10.422798+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:10.422798+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564899,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8737,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3131},{"shards":"2","objects":5156}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11159","reported_seq":11278,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.441474+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:10.441474+0000","last_peered":"2024-09-11T01:19:10.441474+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:10.441474+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8796,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3302},{"shards":"2","objects":5241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11057","reported_seq":14449,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:11.733787+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:11.733787+0000","last_peered":"2024-09-11T01:19:11.733787+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:11.733787+0000","last_undegraded":"2024-09-11T01:19:11.733787+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1633,"num_bytes_recovered":6688768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6764},{"shards":"0,2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11289","reported_seq":11459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.688883+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:11.688883+0000","last_peered":"2024-09-11T01:19:11.688883+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:11.688883+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5185},{"shards":"0,2","objects":3230}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11224","reported_seq":11385,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.733327+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:11.733327+0000","last_peered":"2024-09-11T01:19:11.733327+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:11.733327+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38404096,"num_objects":9376,"num_object_clones":0,"num_object_copies":18752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10993","reported_seq":11145,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.652744+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:11.652744+0000","last_peered":"2024-09-11T01:19:11.652744+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:11.652744+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37458886,"num_objects":9147,"num_object_clones":0,"num_object_copies":18294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7650,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5004},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607859446,"num_objects":148413,"num_object_clones":0,"num_object_copies":296826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41169,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":148413,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":136828,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5933,"num_bytes_recovered":23581155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1277243392,"data_stored":1277083502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692796,"kb_used_data":430632,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93679044,"statfs":{"total":96636764160,"available":95927341056,"internally_reserved":0,"allocated":440967168,"data_stored":440695791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,3,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913789999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591688,"kb_used_data":307320,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93780152,"statfs":{"total":96636764160,"available":96030875648,"internally_reserved":0,"allocated":314695680,"data_stored":314376870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":862836,"kb_used_data":511892,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93509004,"statfs":{"total":96636764160,"available":95753220096,"internally_reserved":0,"allocated":524177408,"data_stored":523906026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,2,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523055104,"data_stored":523018974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314163200,"data_stored":314079642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440025088,"data_stored":439984886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:13.971 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:13.971 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11053", 'reported_seq': 11171, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.757904+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:11.757904+0000', 'last_peered': '2024-09-11T01:19:11.757904+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:11.757904+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37621760, 'num_objects': 9185, 'num_object_clones': 0, 'num_object_copies': 18370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11053, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5088}, {'shards': '0,2', 'objects': 3189}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.971 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:13.971 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'10905", 'reported_seq': 11026, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:10.396569+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:10.396569+0000', 'last_peered': '2024-09-11T01:19:10.396569+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:10.396569+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37437464, 'num_objects': 9141, 'num_object_clones': 0, 'num_object_copies': 18282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9141, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8966, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3124}, {'shards': '2', 'objects': 5157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11034", 'reported_seq': 11202, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.708913+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:11.708913+0000', 'last_peered': '2024-09-11T01:19:11.708913+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:11.708913+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37573091, 'num_objects': 9174, 'num_object_clones': 0, 'num_object_copies': 18348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4097, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9174, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7522, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5078}, {'shards': '0,2', 'objects': 3186}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11381", 'reported_seq': 11546, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.753776+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:11.753776+0000', 'last_peered': '2024-09-11T01:19:11.753776+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:11.753776+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1300", 'ondisk_log_start': "32'1300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38630342, 'num_objects': 9433, 'num_object_clones': 0, 'num_object_copies': 18866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4265, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9433, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8134, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5168}, {'shards': '0,2', 'objects': 3311}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'11254", 'reported_seq': 11412, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.652911+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:11.652911+0000', 'last_peered': '2024-09-11T01:19:11.652911+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:11.652911+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38256640, 'num_objects': 9340, 'num_object_clones': 0, 'num_object_copies': 18680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7618, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5252}, {'shards': '0,2', 'objects': 3152}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'10968", 'reported_seq': 11080, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:10.422798+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:10.422798+0000', 'last_peered': '2024-09-11T01:19:10.422798+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:10.422798+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37564899, 'num_objects': 9172, 'num_object_clones': 0, 'num_object_copies': 18344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9172, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8737, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3131}, {'shards': '2', 'objects': 5156}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:13.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11159", 'reported_seq': 11278, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:10.441474+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:10.441474+0000', 'last_peered': '2024-09-11T01:19:10.441474+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:10.441474+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38514688, 'num_objects': 9403, 'num_object_clones': 0, 'num_object_copies': 18806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9403, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8796, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3302}, {'shards': '2', 'objects': 5241}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11057", 'reported_seq': 14449, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:11.733787+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:11.733787+0000', 'last_peered': '2024-09-11T01:19:11.733787+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:11.733787+0000', 'last_undegraded': '2024-09-11T01:19:11.733787+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38072320, 'num_objects': 9295, 'num_object_clones': 0, 'num_object_copies': 18590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2531, 'num_objects_unfound': 0, 'num_objects_dirty': 9295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7747, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1633, 'num_bytes_recovered': 6688768, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6764}, {'shards': '0,2', 'objects': 2531}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11289", 'reported_seq': 11459, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.688883+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:11.688883+0000', 'last_peered': '2024-09-11T01:19:11.688883+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:11.688883+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38326272, 'num_objects': 9357, 'num_object_clones': 0, 'num_object_copies': 18714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4175, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7682, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5185}, {'shards': '0,2', 'objects': 3230}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'11224", 'reported_seq': 11385, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.733327+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:11.733327+0000', 'last_peered': '2024-09-11T01:19:11.733327+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:11.733327+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38404096, 'num_objects': 9376, 'num_object_clones': 0, 'num_object_copies': 18752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4134, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7759, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5244}, {'shards': '0,2', 'objects': 3226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:13.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'10993", 'reported_seq': 11145, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:11.652744+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:11.652744+0000', 'last_peered': '2024-09-11T01:19:11.652744+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:11.652744+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'900", 'ondisk_log_start': "32'900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37458886, 'num_objects': 9147, 'num_object_clones': 0, 'num_object_copies': 18294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4145, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9147, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7650, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5004}, {'shards': '0,2', 'objects': 3236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:13.974 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:14.692 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:14.692 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41169/296830 objects degraded (13.870%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":355,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":148415,"data_bytes":607859465,"bytes_used":2198855680,"bytes_avail":287711436800,"bytes_total":289910292480,"degraded_objects":41169,"degraded_total":296830,"degraded_ratio":0.13869554964120878,"misplaced_objects":2531,"misplaced_total":296830,"misplaced_ratio":0.0085267661624498867,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":119770,"recovering_keys_per_sec":0,"num_objects_recovered":408,"num_bytes_recovered":1437696,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:14.728 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:14.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:15.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:15.308 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:15.335 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-11T01:19:14.508481+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":607216393,"num_objects":148258,"num_object_clones":0,"num_object_copies":296516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41169,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":148258,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":136987,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5934,"num_bytes_recovered":23581174,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2154748,"kb_used_data":1248952,"kb_used_omap":12,"kb_used_meta":905715,"kb_avail":280960772,"statfs":{"total":289910292480,"available":287703830528,"internally_reserved":0,"allocated":1278926848,"data_stored":1278065279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":927452167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,7,7,6,0,0,0,0,0,9],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18026496,"num_objects":-4401,"num_object_clones":0,"num_object_copies":-8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-31,"num_objects_misplaced":-233,"num_objects_unfound":0,"num_objects_dirty":-4401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4401,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":954368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003977"},"pg_stats":[{"pgid":"2.f","version":"41'11053","reported_seq":11171,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.757904+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:11.757904+0000","last_peered":"2024-09-11T01:19:11.757904+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:11.757904+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37621760,"num_objects":9185,"num_object_clones":0,"num_object_copies":18370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11053,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5088},{"shards":"0,2","objects":3189}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11171","reported_seq":11247,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:10.435171+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:10.435171+0000","last_peered":"2024-09-11T01:19:10.435171+0000","last_clean":"2024-09-11T01:19:10.435171+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:10.435171+0000","last_undegraded":"2024-09-11T01:19:10.435171+0000","last_fullsized":"2024-09-11T01:19:10.435171+0000","mapping_epoch":31,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38302662,"num_objects":9353,"num_object_clones":0,"num_object_copies":18706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9353,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9269,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'10905","reported_seq":11026,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.396569+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:10.396569+0000","last_peered":"2024-09-11T01:19:10.396569+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:10.396569+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37437464,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8966,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3124},{"shards":"2","objects":5157}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11072","reported_seq":11113,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:10.394968+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:10.394968+0000","last_peered":"2024-09-11T01:19:10.394968+0000","last_clean":"2024-09-11T01:19:10.394968+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:10.394968+0000","last_undegraded":"2024-09-11T01:19:10.394968+0000","last_fullsized":"2024-09-11T01:19:10.394968+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38051840,"num_objects":9290,"num_object_clones":0,"num_object_copies":18580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11072,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11088","reported_seq":11205,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.719253+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:11.719253+0000","last_peered":"2024-09-11T01:19:11.719253+0000","last_clean":"2024-09-11T01:19:11.719253+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:11.719253+0000","last_undegraded":"2024-09-11T01:19:11.719253+0000","last_fullsized":"2024-09-11T01:19:11.719253+0000","mapping_epoch":31,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638627,"num_objects":9190,"num_object_clones":0,"num_object_copies":18380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7886,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11034","reported_seq":11202,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.708913+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:11.708913+0000","last_peered":"2024-09-11T01:19:11.708913+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:11.708913+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37573091,"num_objects":9174,"num_object_clones":0,"num_object_copies":18348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4097,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9174,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7522,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5078},{"shards":"0,2","objects":3186}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11381","reported_seq":11546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.753776+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:11.753776+0000","last_peered":"2024-09-11T01:19:11.753776+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:11.753776+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38630342,"num_objects":9433,"num_object_clones":0,"num_object_copies":18866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9433,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8134,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5168},{"shards":"0,2","objects":3311}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11123","reported_seq":11238,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.715764+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:11.715764+0000","last_peered":"2024-09-11T01:19:11.715764+0000","last_clean":"2024-09-11T01:19:11.715764+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:11.715764+0000","last_undegraded":"2024-09-11T01:19:11.715764+0000","last_fullsized":"2024-09-11T01:19:11.715764+0000","mapping_epoch":31,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38096896,"num_objects":9301,"num_object_clones":0,"num_object_copies":18602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8076,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11151","reported_seq":19489,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.309191+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:14.309191+0000","last_peered":"2024-09-11T01:19:14.309191+0000","last_clean":"2024-09-11T01:19:14.309191+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:14.309191+0000","last_undegraded":"2024-09-11T01:19:14.309191+0000","last_fullsized":"2024-09-11T01:19:14.309191+0000","mapping_epoch":40,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37265891,"num_objects":9099,"num_object_clones":0,"num_object_copies":18198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9018,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11254","reported_seq":11412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.652911+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:11.652911+0000","last_peered":"2024-09-11T01:19:11.652911+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:11.652911+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38256640,"num_objects":9340,"num_object_clones":0,"num_object_copies":18680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7618,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5252},{"shards":"0,2","objects":3152}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'10968","reported_seq":11080,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.422798+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:10.422798+0000","last_peered":"2024-09-11T01:19:10.422798+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:10.422798+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37564899,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8737,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3131},{"shards":"2","objects":5156}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11159","reported_seq":11278,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:10.441474+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:10.441474+0000","last_peered":"2024-09-11T01:19:10.441474+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:10.441474+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514688,"num_objects":9403,"num_object_clones":0,"num_object_copies":18806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8796,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3302},{"shards":"2","objects":5241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11057","reported_seq":14449,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:11.733787+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:11.733787+0000","last_peered":"2024-09-11T01:19:11.733787+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:11.733787+0000","last_undegraded":"2024-09-11T01:19:11.733787+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1633,"num_bytes_recovered":6688768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6764},{"shards":"0,2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11289","reported_seq":11459,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.688883+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:11.688883+0000","last_peered":"2024-09-11T01:19:11.688883+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:11.688883+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7682,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5185},{"shards":"0,2","objects":3230}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11224","reported_seq":11385,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.733327+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:11.733327+0000","last_peered":"2024-09-11T01:19:11.733327+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:11.733327+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38404096,"num_objects":9376,"num_object_clones":0,"num_object_copies":18752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7759,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5244},{"shards":"0,2","objects":3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'10993","reported_seq":11145,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.652744+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:11.652744+0000","last_peered":"2024-09-11T01:19:11.652744+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:11.652744+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'900","ondisk_log_start":"32'900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37458886,"num_objects":9147,"num_object_clones":0,"num_object_copies":18294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7650,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5004},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":607216374,"num_objects":148256,"num_object_clones":0,"num_object_copies":296512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41169,"num_objects_misplaced":2531,"num_objects_unfound":0,"num_objects_dirty":148256,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":136985,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5933,"num_bytes_recovered":23581155,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1276407808,"data_stored":1276247918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692796,"kb_used_data":430632,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93679044,"statfs":{"total":96636764160,"available":95927341056,"internally_reserved":0,"allocated":440967168,"data_stored":440695791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":268431022},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,0,3,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599116,"kb_used_data":306428,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93772724,"statfs":{"total":96636764160,"available":96023269376,"internally_reserved":0,"allocated":313782272,"data_stored":313463462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":862836,"kb_used_data":511892,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93509004,"statfs":{"total":96636764160,"available":95753220096,"internally_reserved":0,"allocated":524177408,"data_stored":523906026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,6,2,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523055104,"data_stored":523018974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313327616,"data_stored":313244058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":440025088,"data_stored":439984886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:18.337 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:18.921 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:18.924 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:18.952 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-11T01:19:18.510073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598926089,"num_objects":146234,"num_object_clones":0,"num_object_copies":292468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41157,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":146234,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":139011,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6029,"num_bytes_recovered":23921142,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135352,"kb_used_data":1237892,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280980168,"statfs":{"total":289910292480,"available":287723692032,"internally_reserved":0,"allocated":1267601408,"data_stored":1266739839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,4,6,13,3,0,0,0,0,0,3,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23138304,"num_objects":-5649,"num_object_clones":0,"num_object_copies":-11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-41,"num_objects_misplaced":-316,"num_objects_unfound":0,"num_objects_dirty":-5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5649,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":1294336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004519"},"pg_stats":[{"pgid":"2.f","version":"41'11191","reported_seq":11318,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.791354+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:16.791354+0000","last_peered":"2024-09-11T01:19:16.791354+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:16.791354+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11191,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4953},{"shards":"0,2","objects":3051}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11307","reported_seq":11383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:15.305387+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:15.305387+0000","last_peered":"2024-09-11T01:19:15.305387+0000","last_clean":"2024-09-11T01:19:15.305387+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:15.305387+0000","last_undegraded":"2024-09-11T01:19:15.305387+0000","last_fullsized":"2024-09-11T01:19:15.305387+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37745606,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9405,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11023","reported_seq":11144,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:15.458236+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:15.458236+0000","last_peered":"2024-09-11T01:19:15.458236+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:15.458236+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36954136,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9084,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3006},{"shards":"2","objects":5039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11204","reported_seq":11245,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:15.431301+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:15.431301+0000","last_peered":"2024-09-11T01:19:15.431301+0000","last_clean":"2024-09-11T01:19:15.431301+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:15.431301+0000","last_undegraded":"2024-09-11T01:19:15.431301+0000","last_fullsized":"2024-09-11T01:19:15.431301+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11204,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11220","reported_seq":11337,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.642568+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:16.642568+0000","last_peered":"2024-09-11T01:19:16.642568+0000","last_clean":"2024-09-11T01:19:16.642568+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:16.642568+0000","last_undegraded":"2024-09-11T01:19:16.642568+0000","last_fullsized":"2024-09-11T01:19:16.642568+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097955,"num_objects":9058,"num_object_clones":0,"num_object_copies":18116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8018,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11159","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.765972+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:16.765972+0000","last_peered":"2024-09-11T01:19:16.765972+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:16.765972+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37061091,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7647,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4957},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11525","reported_seq":11689,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.688527+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:16.688527+0000","last_peered":"2024-09-11T01:19:16.688527+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:16.688527+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38040518,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8278,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5024},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11288","reported_seq":11403,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.746073+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:16.746073+0000","last_peered":"2024-09-11T01:19:16.746073+0000","last_clean":"2024-09-11T01:19:16.746073+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:16.746073+0000","last_undegraded":"2024-09-11T01:19:16.746073+0000","last_fullsized":"2024-09-11T01:19:16.746073+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37421056,"num_objects":9136,"num_object_clones":0,"num_object_copies":18272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8241,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11151","reported_seq":19489,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.309191+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:14.309191+0000","last_peered":"2024-09-11T01:19:14.309191+0000","last_clean":"2024-09-11T01:19:14.309191+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:14.309191+0000","last_undegraded":"2024-09-11T01:19:14.309191+0000","last_fullsized":"2024-09-11T01:19:14.309191+0000","mapping_epoch":40,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37265891,"num_objects":9099,"num_object_clones":0,"num_object_copies":18198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9018,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11374","reported_seq":11536,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.776255+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:16.776255+0000","last_peered":"2024-09-11T01:19:16.776255+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:16.776255+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11088","reported_seq":11200,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:15.396133+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:15.396133+0000","last_peered":"2024-09-11T01:19:15.396133+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:15.396133+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37073379,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8857,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3011},{"shards":"2","objects":5036}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11300","reported_seq":11419,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:15.388680+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:15.388680+0000","last_peered":"2024-09-11T01:19:15.388680+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:15.388680+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937152,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8937,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3161},{"shards":"2","objects":5100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11193","reported_seq":14751,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:16.779368+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:16.779368+0000","last_peered":"2024-09-11T01:19:16.779368+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:16.779368+0000","last_undegraded":"2024-09-11T01:19:16.779368+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7883,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1716,"num_bytes_recovered":7028736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6711},{"shards":"0,2","objects":2448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11421","reported_seq":11591,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.790165+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:16.790165+0000","last_peered":"2024-09-11T01:19:16.790165+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:16.790165+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11360","reported_seq":11527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.797609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:16.797609+0000","last_peered":"2024-09-11T01:19:16.797609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:16.797609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7895,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5111},{"shards":"0,2","objects":3090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11142","reported_seq":11294,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.787640+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:16.787640+0000","last_peered":"2024-09-11T01:19:16.787640+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:16.787640+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36848582,"num_objects":8998,"num_object_clones":0,"num_object_copies":17996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7799,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4855},{"shards":"0,2","objects":3087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":598926070,"num_objects":146232,"num_object_clones":0,"num_object_copies":292464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41157,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":146232,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":139009,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6028,"num_bytes_recovered":23921123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265078272,"data_stored":1264918382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692512,"kb_used_data":426188,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93679328,"statfs":{"total":96636764160,"available":95927631872,"internally_reserved":0,"allocated":436416512,"data_stored":436145135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599116,"kb_used_data":306428,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93772724,"statfs":{"total":96636764160,"available":96023269376,"internally_reserved":0,"allocated":313782272,"data_stored":313463462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843724,"kb_used_data":505276,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93528116,"statfs":{"total":96636764160,"available":95772790784,"internally_reserved":0,"allocated":517402624,"data_stored":517131242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,9,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516190208,"data_stored":516154078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313327616,"data_stored":313244058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435560448,"data_stored":435520246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:18.953 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:19.535 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:19.538 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:19.562 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-11T01:19:18.510073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598926089,"num_objects":146234,"num_object_clones":0,"num_object_copies":292468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41157,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":146234,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":139011,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6029,"num_bytes_recovered":23921142,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135352,"kb_used_data":1237892,"kb_used_omap":12,"kb_used_meta":897395,"kb_avail":280980168,"statfs":{"total":289910292480,"available":287723692032,"internally_reserved":0,"allocated":1267601408,"data_stored":1266739839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":918932487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,4,6,13,3,0,0,0,0,0,3,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23138304,"num_objects":-5649,"num_object_clones":0,"num_object_copies":-11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-41,"num_objects_misplaced":-316,"num_objects_unfound":0,"num_objects_dirty":-5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5649,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":357,"num_bytes_recovered":1294336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004519"},"pg_stats":[{"pgid":"2.f","version":"41'11191","reported_seq":11318,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.791354+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:16.791354+0000","last_peered":"2024-09-11T01:19:16.791354+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:16.791354+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11191,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4953},{"shards":"0,2","objects":3051}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11307","reported_seq":11383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:15.305387+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:15.305387+0000","last_peered":"2024-09-11T01:19:15.305387+0000","last_clean":"2024-09-11T01:19:15.305387+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:15.305387+0000","last_undegraded":"2024-09-11T01:19:15.305387+0000","last_fullsized":"2024-09-11T01:19:15.305387+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37745606,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9405,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11023","reported_seq":11144,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:15.458236+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:15.458236+0000","last_peered":"2024-09-11T01:19:15.458236+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:15.458236+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36954136,"num_objects":9023,"num_object_clones":0,"num_object_copies":18046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9084,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3006},{"shards":"2","objects":5039}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11204","reported_seq":11245,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:15.431301+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:15.431301+0000","last_peered":"2024-09-11T01:19:15.431301+0000","last_clean":"2024-09-11T01:19:15.431301+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:15.431301+0000","last_undegraded":"2024-09-11T01:19:15.431301+0000","last_fullsized":"2024-09-11T01:19:15.431301+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37511168,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11204,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11220","reported_seq":11337,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.642568+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:16.642568+0000","last_peered":"2024-09-11T01:19:16.642568+0000","last_clean":"2024-09-11T01:19:16.642568+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:16.642568+0000","last_undegraded":"2024-09-11T01:19:16.642568+0000","last_fullsized":"2024-09-11T01:19:16.642568+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097955,"num_objects":9058,"num_object_clones":0,"num_object_copies":18116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8018,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11159","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.765972+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:16.765972+0000","last_peered":"2024-09-11T01:19:16.765972+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:16.765972+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37061091,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7647,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4957},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11525","reported_seq":11689,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.688527+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:16.688527+0000","last_peered":"2024-09-11T01:19:16.688527+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:16.688527+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38040518,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8278,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5024},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11288","reported_seq":11403,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.746073+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:16.746073+0000","last_peered":"2024-09-11T01:19:16.746073+0000","last_clean":"2024-09-11T01:19:16.746073+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:16.746073+0000","last_undegraded":"2024-09-11T01:19:16.746073+0000","last_fullsized":"2024-09-11T01:19:16.746073+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37421056,"num_objects":9136,"num_object_clones":0,"num_object_copies":18272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8241,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892044+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892044+0000","last_peered":"2024-09-11T01:17:37.892044+0000","last_clean":"2024-09-11T01:17:37.892044+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892044+0000","last_undegraded":"2024-09-11T01:17:37.892044+0000","last_fullsized":"2024-09-11T01:17:37.892044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11151","reported_seq":19489,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.309191+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:14.309191+0000","last_peered":"2024-09-11T01:19:14.309191+0000","last_clean":"2024-09-11T01:19:14.309191+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:14.309191+0000","last_undegraded":"2024-09-11T01:19:14.309191+0000","last_fullsized":"2024-09-11T01:19:14.309191+0000","mapping_epoch":40,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37265891,"num_objects":9099,"num_object_clones":0,"num_object_copies":18198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9018,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11374","reported_seq":11536,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.776255+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:16.776255+0000","last_peered":"2024-09-11T01:19:16.776255+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:16.776255+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11088","reported_seq":11200,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:15.396133+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:15.396133+0000","last_peered":"2024-09-11T01:19:15.396133+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:15.396133+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1000","ondisk_log_start":"32'1000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37073379,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8857,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3011},{"shards":"2","objects":5036}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11300","reported_seq":11419,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:15.388680+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:15.388680+0000","last_peered":"2024-09-11T01:19:15.388680+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:15.388680+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37937152,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8937,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3161},{"shards":"2","objects":5100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11193","reported_seq":14751,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:16.779368+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:16.779368+0000","last_peered":"2024-09-11T01:19:16.779368+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:16.779368+0000","last_undegraded":"2024-09-11T01:19:16.779368+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7883,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1716,"num_bytes_recovered":7028736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6711},{"shards":"0,2","objects":2448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11421","reported_seq":11591,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.790165+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:16.790165+0000","last_peered":"2024-09-11T01:19:16.790165+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:16.790165+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11360","reported_seq":11527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.797609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:16.797609+0000","last_peered":"2024-09-11T01:19:16.797609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:16.797609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7895,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5111},{"shards":"0,2","objects":3090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11142","reported_seq":11294,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.787640+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:16.787640+0000","last_peered":"2024-09-11T01:19:16.787640+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:16.787640+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36848582,"num_objects":8998,"num_object_clones":0,"num_object_copies":17996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7799,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4855},{"shards":"0,2","objects":3087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":598926070,"num_objects":146232,"num_object_clones":0,"num_object_copies":292464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41157,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":146232,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":139009,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6028,"num_bytes_recovered":23921123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1265078272,"data_stored":1264918382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":692512,"kb_used_data":426188,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93679328,"statfs":{"total":96636764160,"available":95927631872,"internally_reserved":0,"allocated":436416512,"data_stored":436145135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":272690862},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,1,0,0,0,0,0,0,1],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599116,"kb_used_data":306428,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93772724,"statfs":{"total":96636764160,"available":96023269376,"internally_reserved":0,"allocated":313782272,"data_stored":313463462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843724,"kb_used_data":505276,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93528116,"statfs":{"total":96636764160,"available":95772790784,"internally_reserved":0,"allocated":517402624,"data_stored":517131242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,9,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516190208,"data_stored":516154078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313327616,"data_stored":313244058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435560448,"data_stored":435520246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:19.563 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:19.563 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11191", 'reported_seq': 11318, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.791354+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:16.791354+0000', 'last_peered': '2024-09-11T01:19:16.791354+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:16.791354+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37056512, 'num_objects': 9047, 'num_object_clones': 0, 'num_object_copies': 18094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4094, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11191, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4953}, {'shards': '0,2', 'objects': 3051}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.563 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:19.563 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11023", 'reported_seq': 11144, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:15.458236+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:15.458236+0000', 'last_peered': '2024-09-11T01:19:15.458236+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:15.458236+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36954136, 'num_objects': 9023, 'num_object_clones': 0, 'num_object_copies': 18046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9023, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9084, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3006}, {'shards': '2', 'objects': 5039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11159", 'reported_seq': 11335, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.765972+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:16.765972+0000', 'last_peered': '2024-09-11T01:19:16.765972+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:16.765972+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37061091, 'num_objects': 9049, 'num_object_clones': 0, 'num_object_copies': 18098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4093, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9049, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7647, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4957}, {'shards': '0,2', 'objects': 3061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11525", 'reported_seq': 11689, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.688527+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:16.688527+0000', 'last_peered': '2024-09-11T01:19:16.688527+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:16.688527+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38040518, 'num_objects': 9289, 'num_object_clones': 0, 'num_object_copies': 18578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4265, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9289, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8278, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5024}, {'shards': '0,2', 'objects': 3167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'11374", 'reported_seq': 11536, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.776255+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:16.776255+0000', 'last_peered': '2024-09-11T01:19:16.776255+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:16.776255+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'1300", 'ondisk_log_start': "32'1300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37765120, 'num_objects': 9220, 'num_object_clones': 0, 'num_object_copies': 18440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7738, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5134}, {'shards': '0,2', 'objects': 3032}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'11088", 'reported_seq': 11200, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:15.396133+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:15.396133+0000', 'last_peered': '2024-09-11T01:19:15.396133+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:15.396133+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'1000", 'ondisk_log_start': "32'1000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37073379, 'num_objects': 9052, 'num_object_clones': 0, 'num_object_copies': 18104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9052, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8857, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3011}, {'shards': '2', 'objects': 5036}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:19.564 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11300", 'reported_seq': 11419, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:15.388680+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:15.388680+0000', 'last_peered': '2024-09-11T01:19:15.388680+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:15.388680+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37937152, 'num_objects': 9262, 'num_object_clones': 0, 'num_object_copies': 18524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9262, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8937, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3161}, {'shards': '2', 'objects': 5100}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11193", 'reported_seq': 14751, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:16.779368+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:16.779368+0000', 'last_peered': '2024-09-11T01:19:16.779368+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:16.779368+0000', 'last_undegraded': '2024-09-11T01:19:16.779368+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37515264, 'num_objects': 9159, 'num_object_clones': 0, 'num_object_copies': 18318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2448, 'num_objects_unfound': 0, 'num_objects_dirty': 9159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7883, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1716, 'num_bytes_recovered': 7028736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6711}, {'shards': '0,2', 'objects': 2448}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11421", 'reported_seq': 11591, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.790165+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:16.790165+0000', 'last_peered': '2024-09-11T01:19:16.790165+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:16.790165+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37785600, 'num_objects': 9225, 'num_object_clones': 0, 'num_object_copies': 18450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4175, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9225, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7814, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5053}, {'shards': '0,2', 'objects': 3098}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'11360", 'reported_seq': 11527, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.797609+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:16.797609+0000', 'last_peered': '2024-09-11T01:19:16.797609+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:16.797609+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'1300", 'ondisk_log_start': "32'1300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37847040, 'num_objects': 9240, 'num_object_clones': 0, 'num_object_copies': 18480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4131, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7895, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 19, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5111}, {'shards': '0,2', 'objects': 3090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:19.565 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'11142", 'reported_seq': 11294, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.787640+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:16.787640+0000', 'last_peered': '2024-09-11T01:19:16.787640+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:16.787640+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36848582, 'num_objects': 8998, 'num_object_clones': 0, 'num_object_copies': 17996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4145, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8998, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7799, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4855}, {'shards': '0,2', 'objects': 3087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:19.566 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:20.287 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:20.287 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41157/292468 objects degraded (14.072%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":360,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":146234,"data_bytes":598926089,"bytes_used":2186600448,"bytes_avail":287723692032,"bytes_total":289910292480,"degraded_objects":41157,"degraded_total":292468,"degraded_ratio":0.14072308765403393,"misplaced_objects":2448,"misplaced_total":292468,"misplaced_ratio":0.0083701464775633571,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":107820,"recovering_keys_per_sec":0,"num_objects_recovered":357,"num_bytes_recovered":1294336,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":470},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:20.315 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:20.315 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:20.898 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:20.899 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:20.921 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-11T01:19:20.510741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595579657,"num_objects":145417,"num_object_clones":0,"num_object_copies":290834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41157,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":145417,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":139828,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6029,"num_bytes_recovered":23921142,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139968,"kb_used_data":1232784,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280975552,"statfs":{"total":289910292480,"available":287718965248,"internally_reserved":0,"allocated":1262370816,"data_stored":1261509247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":928893959},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,3,5,15,4,0,0,0,0,0,3,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21082112,"num_objects":-5147,"num_object_clones":0,"num_object_copies":-10294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-20,"num_objects_misplaced":-189,"num_objects_unfound":0,"num_objects_dirty":-5147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5147,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":774144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004499"},"pg_stats":[{"pgid":"2.f","version":"41'11191","reported_seq":11318,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.791354+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:16.791354+0000","last_peered":"2024-09-11T01:19:16.791354+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:16.791354+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37056512,"num_objects":9047,"num_object_clones":0,"num_object_copies":18094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11191,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4953},{"shards":"0,2","objects":3051}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11461","reported_seq":11537,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.445669+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:20.445669+0000","last_peered":"2024-09-11T01:19:20.445669+0000","last_clean":"2024-09-11T01:19:20.445669+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:20.445669+0000","last_undegraded":"2024-09-11T01:19:20.445669+0000","last_fullsized":"2024-09-11T01:19:20.445669+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9559,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11151","reported_seq":11273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.456700+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:20.456700+0000","last_peered":"2024-09-11T01:19:20.456700+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:20.456700+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429848,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9212,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":4911}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11342","reported_seq":11383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.340550+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:20.340550+0000","last_peered":"2024-09-11T01:19:20.340550+0000","last_clean":"2024-09-11T01:19:20.340550+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:20.340550+0000","last_undegraded":"2024-09-11T01:19:20.340550+0000","last_fullsized":"2024-09-11T01:19:20.340550+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11220","reported_seq":11337,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.642568+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:16.642568+0000","last_peered":"2024-09-11T01:19:16.642568+0000","last_clean":"2024-09-11T01:19:16.642568+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:16.642568+0000","last_undegraded":"2024-09-11T01:19:16.642568+0000","last_fullsized":"2024-09-11T01:19:16.642568+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37097955,"num_objects":9058,"num_object_clones":0,"num_object_copies":18116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9058,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8018,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11159","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.765972+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:16.765972+0000","last_peered":"2024-09-11T01:19:16.765972+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:16.765972+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37061091,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7647,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4957},{"shards":"0,2","objects":3061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11525","reported_seq":11689,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.688527+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:16.688527+0000","last_peered":"2024-09-11T01:19:16.688527+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:16.688527+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38040518,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8278,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5024},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11288","reported_seq":11403,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.746073+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:16.746073+0000","last_peered":"2024-09-11T01:19:16.746073+0000","last_clean":"2024-09-11T01:19:16.746073+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:16.746073+0000","last_undegraded":"2024-09-11T01:19:16.746073+0000","last_fullsized":"2024-09-11T01:19:16.746073+0000","mapping_epoch":31,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37421056,"num_objects":9136,"num_object_clones":0,"num_object_copies":18272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8241,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11282","reported_seq":19620,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.150466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:19.150466+0000","last_peered":"2024-09-11T01:19:19.150466+0000","last_clean":"2024-09-11T01:19:19.150466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:19.150466+0000","last_undegraded":"2024-09-11T01:19:19.150466+0000","last_fullsized":"2024-09-11T01:19:19.150466+0000","mapping_epoch":40,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36729315,"num_objects":8968,"num_object_clones":0,"num_object_copies":17936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9149,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11374","reported_seq":11536,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.776255+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:16.776255+0000","last_peered":"2024-09-11T01:19:16.776255+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:16.776255+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37765120,"num_objects":9220,"num_object_clones":0,"num_object_copies":18440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7738,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3032}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11223","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.400679+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:20.400679+0000","last_peered":"2024-09-11T01:19:20.400679+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:20.400679+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36520419,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8992,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2876},{"shards":"2","objects":4901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11431","reported_seq":11550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.460328+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:20.460328+0000","last_peered":"2024-09-11T01:19:20.460328+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:20.460328+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9068,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3030},{"shards":"2","objects":4969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11193","reported_seq":14751,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:16.779368+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:16.779368+0000","last_peered":"2024-09-11T01:19:16.779368+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:16.779368+0000","last_undegraded":"2024-09-11T01:19:16.779368+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37515264,"num_objects":9159,"num_object_clones":0,"num_object_copies":18318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":9159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7883,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1716,"num_bytes_recovered":7028736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6711},{"shards":"0,2","objects":2448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11421","reported_seq":11591,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.790165+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:16.790165+0000","last_peered":"2024-09-11T01:19:16.790165+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:16.790165+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37785600,"num_objects":9225,"num_object_clones":0,"num_object_copies":18450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7814,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3098}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11360","reported_seq":11527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.797609+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:16.797609+0000","last_peered":"2024-09-11T01:19:16.797609+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:16.797609+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37847040,"num_objects":9240,"num_object_clones":0,"num_object_copies":18480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7895,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5111},{"shards":"0,2","objects":3090}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11142","reported_seq":11294,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.787640+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:16.787640+0000","last_peered":"2024-09-11T01:19:16.787640+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:16.787640+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36848582,"num_objects":8998,"num_object_clones":0,"num_object_copies":17996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8998,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7799,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4855},{"shards":"0,2","objects":3087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":595579638,"num_objects":145415,"num_object_clones":0,"num_object_copies":290830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41157,"num_objects_misplaced":2448,"num_objects_unfound":0,"num_objects_dirty":145415,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":139826,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6028,"num_bytes_recovered":23921123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1259831296,"data_stored":1259671406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710276,"kb_used_data":421748,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93661564,"statfs":{"total":96636764160,"available":95909441536,"internally_reserved":0,"allocated":431869952,"data_stored":431598575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,0,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790001,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585968,"kb_used_data":305760,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93785872,"statfs":{"total":96636764160,"available":96036732928,"internally_reserved":0,"allocated":313098240,"data_stored":312779430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843724,"kb_used_data":505276,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93528116,"statfs":{"total":96636764160,"available":95772790784,"internally_reserved":0,"allocated":517402624,"data_stored":517131242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,2,9,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516190208,"data_stored":516154078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312610816,"data_stored":312527258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431030272,"data_stored":430990070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:23.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:24.500 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:24.500 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:24.526 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-11T01:19:22.512700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590152457,"num_objects":144092,"num_object_clones":0,"num_object_copies":288184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41150,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":144092,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":141153,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6143,"num_bytes_recovered":24359414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2137768,"kb_used_data":1226424,"kb_used_omap":12,"kb_used_meta":911283,"kb_avail":280977752,"statfs":{"total":289910292480,"available":287721218048,"internally_reserved":0,"allocated":1255858176,"data_stored":1254996607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":933153799},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,5,10,5,4,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23257088,"num_objects":-5678,"num_object_clones":0,"num_object_copies":-11356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-27,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":-5678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5678,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005590"},"pg_stats":[{"pgid":"2.f","version":"41'11308","reported_seq":11435,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.844369+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:21.844369+0000","last_peered":"2024-09-11T01:19:21.844369+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:21.844369+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4836},{"shards":"0,2","objects":2934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11461","reported_seq":11537,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.445669+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:20.445669+0000","last_peered":"2024-09-11T01:19:20.445669+0000","last_clean":"2024-09-11T01:19:20.445669+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:20.445669+0000","last_undegraded":"2024-09-11T01:19:20.445669+0000","last_fullsized":"2024-09-11T01:19:20.445669+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9559,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11151","reported_seq":11273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.456700+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:20.456700+0000","last_peered":"2024-09-11T01:19:20.456700+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:20.456700+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429848,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9212,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":4911}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11342","reported_seq":11383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.340550+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:20.340550+0000","last_peered":"2024-09-11T01:19:20.340550+0000","last_clean":"2024-09-11T01:19:20.340550+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:20.340550+0000","last_undegraded":"2024-09-11T01:19:20.340550+0000","last_fullsized":"2024-09-11T01:19:20.340550+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11353","reported_seq":11470,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.827063+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:21.827063+0000","last_peered":"2024-09-11T01:19:21.827063+0000","last_clean":"2024-09-11T01:19:21.827063+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:21.827063+0000","last_undegraded":"2024-09-11T01:19:21.827063+0000","last_fullsized":"2024-09-11T01:19:21.827063+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36553187,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8151,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11300","reported_seq":11476,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.837553+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:21.837553+0000","last_peered":"2024-09-11T01:19:21.837553+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:21.837553+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36483555,"num_objects":8908,"num_object_clones":0,"num_object_copies":17816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7788,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":2920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11653","reported_seq":11818,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.843089+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:21.843089+0000","last_peered":"2024-09-11T01:19:21.843089+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:21.843089+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516230,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8406,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4896},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11445","reported_seq":11560,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.813947+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:21.813947+0000","last_peered":"2024-09-11T01:19:21.813947+0000","last_clean":"2024-09-11T01:19:21.813947+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:21.813947+0000","last_undegraded":"2024-09-11T01:19:21.813947+0000","last_fullsized":"2024-09-11T01:19:21.813947+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36777984,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11282","reported_seq":19620,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.150466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:19.150466+0000","last_peered":"2024-09-11T01:19:19.150466+0000","last_clean":"2024-09-11T01:19:19.150466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:19.150466+0000","last_undegraded":"2024-09-11T01:19:19.150466+0000","last_fullsized":"2024-09-11T01:19:19.150466+0000","mapping_epoch":40,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36729315,"num_objects":8968,"num_object_clones":0,"num_object_copies":17936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9149,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11497","reported_seq":11662,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.790582+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:21.790582+0000","last_peered":"2024-09-11T01:19:21.790582+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:21.790582+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7861,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5012},{"shards":"0,2","objects":2909}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11223","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.400679+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:20.400679+0000","last_peered":"2024-09-11T01:19:20.400679+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:20.400679+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36520419,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8992,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2876},{"shards":"2","objects":4901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11431","reported_seq":11550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.460328+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:20.460328+0000","last_peered":"2024-09-11T01:19:20.460328+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:20.460328+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9068,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3030},{"shards":"2","objects":4969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11332","reported_seq":15104,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:21.883649+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:21.883649+0000","last_peered":"2024-09-11T01:19:21.883649+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:21.883649+0000","last_undegraded":"2024-09-11T01:19:21.883649+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8022,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1823,"num_bytes_recovered":7467008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6679},{"shards":"0,2","objects":2341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11545","reported_seq":11719,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.720957+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:21.720957+0000","last_peered":"2024-09-11T01:19:21.720957+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:21.720957+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":2974}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11484","reported_seq":11657,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.729548+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:21.729548+0000","last_peered":"2024-09-11T01:19:21.729548+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:21.729548+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8019,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4990},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11281","reported_seq":11436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.804320+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:21.804320+0000","last_peered":"2024-09-11T01:19:21.804320+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:21.804320+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36279238,"num_objects":8859,"num_object_clones":0,"num_object_copies":17718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7938,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":590152438,"num_objects":144090,"num_object_clones":0,"num_object_copies":288180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41150,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":144090,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":141151,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6142,"num_bytes_recovered":24359395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1253236736,"data_stored":1253076846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710276,"kb_used_data":421748,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93661564,"statfs":{"total":96636764160,"available":95909441536,"internally_reserved":0,"allocated":431869952,"data_stored":431598575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,0,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790001,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585968,"kb_used_data":305760,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93785872,"statfs":{"total":96636764160,"available":96036732928,"internally_reserved":0,"allocated":313098240,"data_stored":312779430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841524,"kb_used_data":498916,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93530316,"statfs":{"total":96636764160,"available":95775043584,"internally_reserved":0,"allocated":510889984,"data_stored":510618602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509595648,"data_stored":509559518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312610816,"data_stored":312527258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431030272,"data_stored":430990070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:24.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:25.129 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:25.130 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:25.153 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-11T01:19:24.513658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":589554441,"num_objects":143946,"num_object_clones":0,"num_object_copies":287892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41150,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":143946,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":141299,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6143,"num_bytes_recovered":24359414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2145356,"kb_used_data":1225692,"kb_used_omap":12,"kb_used_meta":919603,"kb_avail":280970164,"statfs":{"total":289910292480,"available":287713447936,"internally_reserved":0,"allocated":1255108608,"data_stored":1254247039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":941673479},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,4,12,3,4,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18305024,"num_objects":-4469,"num_object_clones":0,"num_object_copies":-8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-19,"num_objects_misplaced":-190,"num_objects_unfound":0,"num_objects_dirty":-4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005894"},"pg_stats":[{"pgid":"2.f","version":"41'11308","reported_seq":11435,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.844369+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:21.844369+0000","last_peered":"2024-09-11T01:19:21.844369+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:21.844369+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4836},{"shards":"0,2","objects":2934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11461","reported_seq":11537,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.445669+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:20.445669+0000","last_peered":"2024-09-11T01:19:20.445669+0000","last_clean":"2024-09-11T01:19:20.445669+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:20.445669+0000","last_undegraded":"2024-09-11T01:19:20.445669+0000","last_fullsized":"2024-09-11T01:19:20.445669+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9559,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11151","reported_seq":11273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.456700+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:20.456700+0000","last_peered":"2024-09-11T01:19:20.456700+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:20.456700+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429848,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9212,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":4911}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11342","reported_seq":11383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.340550+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:20.340550+0000","last_peered":"2024-09-11T01:19:20.340550+0000","last_clean":"2024-09-11T01:19:20.340550+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:20.340550+0000","last_undegraded":"2024-09-11T01:19:20.340550+0000","last_fullsized":"2024-09-11T01:19:20.340550+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11353","reported_seq":11470,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.827063+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:21.827063+0000","last_peered":"2024-09-11T01:19:21.827063+0000","last_clean":"2024-09-11T01:19:21.827063+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:21.827063+0000","last_undegraded":"2024-09-11T01:19:21.827063+0000","last_fullsized":"2024-09-11T01:19:21.827063+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36553187,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8151,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11300","reported_seq":11476,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.837553+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:21.837553+0000","last_peered":"2024-09-11T01:19:21.837553+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:21.837553+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36483555,"num_objects":8908,"num_object_clones":0,"num_object_copies":17816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7788,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":2920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11653","reported_seq":11818,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.843089+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:21.843089+0000","last_peered":"2024-09-11T01:19:21.843089+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:21.843089+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516230,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8406,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4896},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11445","reported_seq":11560,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.813947+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:21.813947+0000","last_peered":"2024-09-11T01:19:21.813947+0000","last_clean":"2024-09-11T01:19:21.813947+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:21.813947+0000","last_undegraded":"2024-09-11T01:19:21.813947+0000","last_fullsized":"2024-09-11T01:19:21.813947+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36777984,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11428","reported_seq":19766,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:24.349423+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:24.349423+0000","last_peered":"2024-09-11T01:19:24.349423+0000","last_clean":"2024-09-11T01:19:24.349423+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:24.349423+0000","last_undegraded":"2024-09-11T01:19:24.349423+0000","last_fullsized":"2024-09-11T01:19:24.349423+0000","mapping_epoch":40,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36131299,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9295,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11497","reported_seq":11662,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.790582+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:21.790582+0000","last_peered":"2024-09-11T01:19:21.790582+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:21.790582+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7861,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5012},{"shards":"0,2","objects":2909}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11223","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.400679+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:20.400679+0000","last_peered":"2024-09-11T01:19:20.400679+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:20.400679+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36520419,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8992,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2876},{"shards":"2","objects":4901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11431","reported_seq":11550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.460328+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:20.460328+0000","last_peered":"2024-09-11T01:19:20.460328+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:20.460328+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9068,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3030},{"shards":"2","objects":4969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11332","reported_seq":15104,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:21.883649+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:21.883649+0000","last_peered":"2024-09-11T01:19:21.883649+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:21.883649+0000","last_undegraded":"2024-09-11T01:19:21.883649+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8022,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1823,"num_bytes_recovered":7467008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6679},{"shards":"0,2","objects":2341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11545","reported_seq":11719,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.720957+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:21.720957+0000","last_peered":"2024-09-11T01:19:21.720957+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:21.720957+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":2974}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11484","reported_seq":11657,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.729548+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:21.729548+0000","last_peered":"2024-09-11T01:19:21.729548+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:21.729548+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8019,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4990},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11281","reported_seq":11436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.804320+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:21.804320+0000","last_peered":"2024-09-11T01:19:21.804320+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:21.804320+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36279238,"num_objects":8859,"num_object_clones":0,"num_object_copies":17718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7938,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":589554422,"num_objects":143944,"num_object_clones":0,"num_object_copies":287888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41150,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":143944,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":141297,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6142,"num_bytes_recovered":24359395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1252487168,"data_stored":1252327278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710276,"kb_used_data":421748,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93661564,"statfs":{"total":96636764160,"available":95909441536,"internally_reserved":0,"allocated":431869952,"data_stored":431598575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,0,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593556,"kb_used_data":305028,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93778284,"statfs":{"total":96636764160,"available":96028962816,"internally_reserved":0,"allocated":312348672,"data_stored":312029862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841524,"kb_used_data":498916,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93530316,"statfs":{"total":96636764160,"available":95775043584,"internally_reserved":0,"allocated":510889984,"data_stored":510618602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509595648,"data_stored":509559518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311861248,"data_stored":311777690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431030272,"data_stored":430990070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:25.154 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:25.154 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11308", 'reported_seq': 11435, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.844369+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:21.844369+0000', 'last_peered': '2024-09-11T01:19:21.844369+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:21.844369+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1300", 'ondisk_log_start': "32'1300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36577280, 'num_objects': 8930, 'num_object_clones': 0, 'num_object_copies': 17860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4094, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11308, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4836}, {'shards': '0,2', 'objects': 2934}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.154 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:25.154 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11151", 'reported_seq': 11273, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:20.456700+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:20.456700+0000', 'last_peered': '2024-09-11T01:19:20.456700+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:20.456700+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1100", 'ondisk_log_start': "32'1100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36429848, 'num_objects': 8895, 'num_object_clones': 0, 'num_object_copies': 17790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8895, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9212, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2878}, {'shards': '2', 'objects': 4911}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11300", 'reported_seq': 11476, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.837553+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:21.837553+0000', 'last_peered': '2024-09-11T01:19:21.837553+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:21.837553+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36483555, 'num_objects': 8908, 'num_object_clones': 0, 'num_object_copies': 17816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4093, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8908, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7788, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4816}, {'shards': '0,2', 'objects': 2920}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11653", 'reported_seq': 11818, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.843089+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:21.843089+0000', 'last_peered': '2024-09-11T01:19:21.843089+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:21.843089+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1600", 'ondisk_log_start': "32'1600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37516230, 'num_objects': 9161, 'num_object_clones': 0, 'num_object_copies': 18322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4265, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9161, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8406, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4896}, {'shards': '0,2', 'objects': 3039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'11497", 'reported_seq': 11662, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.790582+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:21.790582+0000', 'last_peered': '2024-09-11T01:19:21.790582+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:21.790582+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37261312, 'num_objects': 9097, 'num_object_clones': 0, 'num_object_copies': 18194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4087, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7861, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5012}, {'shards': '0,2', 'objects': 2909}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'11223", 'reported_seq': 11335, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:20.400679+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:20.400679+0000', 'last_peered': '2024-09-11T01:19:20.400679+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:20.400679+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36520419, 'num_objects': 8917, 'num_object_clones': 0, 'num_object_copies': 17834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8917, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8992, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2876}, {'shards': '2', 'objects': 4901}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11431", 'reported_seq': 11550, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:20.460328+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:20.460328+0000', 'last_peered': '2024-09-11T01:19:20.460328+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:20.460328+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37400576, 'num_objects': 9131, 'num_object_clones': 0, 'num_object_copies': 18262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9068, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3030}, {'shards': '2', 'objects': 4969}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:25.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11332", 'reported_seq': 15104, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:21.883649+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:21.883649+0000', 'last_peered': '2024-09-11T01:19:21.883649+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:21.883649+0000', 'last_undegraded': '2024-09-11T01:19:21.883649+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1300", 'ondisk_log_start': "32'1300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36945920, 'num_objects': 9020, 'num_object_clones': 0, 'num_object_copies': 18040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2341, 'num_objects_unfound': 0, 'num_objects_dirty': 9020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8022, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1823, 'num_bytes_recovered': 7467008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6679}, {'shards': '0,2', 'objects': 2341}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11545", 'reported_seq': 11719, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.720957+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:21.720957+0000', 'last_peered': '2024-09-11T01:19:21.720957+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:21.720957+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37277696, 'num_objects': 9101, 'num_object_clones': 0, 'num_object_copies': 18202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4173, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7938, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4931}, {'shards': '0,2', 'objects': 2974}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'11484", 'reported_seq': 11657, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.729548+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:21.729548+0000', 'last_peered': '2024-09-11T01:19:21.729548+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:21.729548+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37339136, 'num_objects': 9116, 'num_object_clones': 0, 'num_object_copies': 18232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4128, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9116, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8019, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4990}, {'shards': '0,2', 'objects': 2966}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:25.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'11281", 'reported_seq': 11436, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.804320+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:21.804320+0000', 'last_peered': '2024-09-11T01:19:21.804320+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:21.804320+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36279238, 'num_objects': 8859, 'num_object_clones': 0, 'num_object_copies': 17718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4144, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8859, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7938, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4717}, {'shards': '0,2', 'objects': 2948}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.156 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:25.845 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:25.846 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41150/287892 objects degraded (14.294%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":366,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143946,"data_bytes":589554441,"bytes_used":2196844544,"bytes_avail":287713447936,"bytes_total":289910292480,"degraded_objects":41150,"degraded_total":287892,"degraded_ratio":0.14293554527392216,"misplaced_objects":2341,"misplaced_total":287892,"misplaced_ratio":0.0081315215428007723,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":64821,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":778240,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:25.876 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:25.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:26.462 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:26.463 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:26.484 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-11T01:19:24.513658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":589554441,"num_objects":143946,"num_object_clones":0,"num_object_copies":287892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41150,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":143946,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":141299,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6143,"num_bytes_recovered":24359414,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2145356,"kb_used_data":1225692,"kb_used_omap":12,"kb_used_meta":919603,"kb_avail":280970164,"statfs":{"total":289910292480,"available":287713447936,"internally_reserved":0,"allocated":1255108608,"data_stored":1254247039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":941673479},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,4,12,3,4,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18305024,"num_objects":-4469,"num_object_clones":0,"num_object_copies":-8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-19,"num_objects_misplaced":-190,"num_objects_unfound":0,"num_objects_dirty":-4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4469,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005894"},"pg_stats":[{"pgid":"2.f","version":"41'11308","reported_seq":11435,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.844369+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:21.844369+0000","last_peered":"2024-09-11T01:19:21.844369+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:21.844369+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36577280,"num_objects":8930,"num_object_clones":0,"num_object_copies":17860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4836},{"shards":"0,2","objects":2934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11461","reported_seq":11537,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.445669+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:20.445669+0000","last_peered":"2024-09-11T01:19:20.445669+0000","last_clean":"2024-09-11T01:19:20.445669+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:20.445669+0000","last_undegraded":"2024-09-11T01:19:20.445669+0000","last_fullsized":"2024-09-11T01:19:20.445669+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37114822,"num_objects":9063,"num_object_clones":0,"num_object_copies":18126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9063,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9559,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11151","reported_seq":11273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.456700+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:20.456700+0000","last_peered":"2024-09-11T01:19:20.456700+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:20.456700+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1100","ondisk_log_start":"32'1100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36429848,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9212,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2878},{"shards":"2","objects":4911}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11342","reported_seq":11383,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:20.340550+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:20.340550+0000","last_peered":"2024-09-11T01:19:20.340550+0000","last_clean":"2024-09-11T01:19:20.340550+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:20.340550+0000","last_undegraded":"2024-09-11T01:19:20.340550+0000","last_fullsized":"2024-09-11T01:19:20.340550+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11353","reported_seq":11470,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.827063+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:21.827063+0000","last_peered":"2024-09-11T01:19:21.827063+0000","last_clean":"2024-09-11T01:19:21.827063+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:21.827063+0000","last_undegraded":"2024-09-11T01:19:21.827063+0000","last_fullsized":"2024-09-11T01:19:21.827063+0000","mapping_epoch":31,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36553187,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8151,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11300","reported_seq":11476,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.837553+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:21.837553+0000","last_peered":"2024-09-11T01:19:21.837553+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:21.837553+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36483555,"num_objects":8908,"num_object_clones":0,"num_object_copies":17816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8908,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7788,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":2920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11653","reported_seq":11818,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.843089+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:21.843089+0000","last_peered":"2024-09-11T01:19:21.843089+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:21.843089+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37516230,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8406,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4896},{"shards":"0,2","objects":3039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11445","reported_seq":11560,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.813947+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:21.813947+0000","last_peered":"2024-09-11T01:19:21.813947+0000","last_clean":"2024-09-11T01:19:21.813947+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:21.813947+0000","last_undegraded":"2024-09-11T01:19:21.813947+0000","last_fullsized":"2024-09-11T01:19:21.813947+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36777984,"num_objects":8979,"num_object_clones":0,"num_object_copies":17958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8398,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11428","reported_seq":19766,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:24.349423+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:24.349423+0000","last_peered":"2024-09-11T01:19:24.349423+0000","last_clean":"2024-09-11T01:19:24.349423+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:24.349423+0000","last_undegraded":"2024-09-11T01:19:24.349423+0000","last_fullsized":"2024-09-11T01:19:24.349423+0000","mapping_epoch":40,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36131299,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9295,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11497","reported_seq":11662,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.790582+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:21.790582+0000","last_peered":"2024-09-11T01:19:21.790582+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:21.790582+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7861,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5012},{"shards":"0,2","objects":2909}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11223","reported_seq":11335,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.400679+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:20.400679+0000","last_peered":"2024-09-11T01:19:20.400679+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:20.400679+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36520419,"num_objects":8917,"num_object_clones":0,"num_object_copies":17834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8992,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2876},{"shards":"2","objects":4901}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11431","reported_seq":11550,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:20.460328+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:20.460328+0000","last_peered":"2024-09-11T01:19:20.460328+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:20.460328+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37400576,"num_objects":9131,"num_object_clones":0,"num_object_copies":18262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9068,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3030},{"shards":"2","objects":4969}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11332","reported_seq":15104,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:21.883649+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:21.883649+0000","last_peered":"2024-09-11T01:19:21.883649+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:21.883649+0000","last_undegraded":"2024-09-11T01:19:21.883649+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36945920,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8022,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1823,"num_bytes_recovered":7467008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6679},{"shards":"0,2","objects":2341}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11545","reported_seq":11719,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.720957+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:21.720957+0000","last_peered":"2024-09-11T01:19:21.720957+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:21.720957+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4173,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7938,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":2974}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11484","reported_seq":11657,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.729548+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:21.729548+0000","last_peered":"2024-09-11T01:19:21.729548+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:21.729548+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37339136,"num_objects":9116,"num_object_clones":0,"num_object_copies":18232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8019,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4990},{"shards":"0,2","objects":2966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11281","reported_seq":11436,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.804320+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:21.804320+0000","last_peered":"2024-09-11T01:19:21.804320+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:21.804320+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36279238,"num_objects":8859,"num_object_clones":0,"num_object_copies":17718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7938,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4717},{"shards":"0,2","objects":2948}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":589554422,"num_objects":143944,"num_object_clones":0,"num_object_copies":287888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41150,"num_objects_misplaced":2341,"num_objects_unfound":0,"num_objects_dirty":143944,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":141297,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6142,"num_bytes_recovered":24359395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1252487168,"data_stored":1252327278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710276,"kb_used_data":421748,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93661564,"statfs":{"total":96636764160,"available":95909441536,"internally_reserved":0,"allocated":431869952,"data_stored":431598575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,4,0,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593556,"kb_used_data":305028,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93778284,"statfs":{"total":96636764160,"available":96028962816,"internally_reserved":0,"allocated":312348672,"data_stored":312029862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841524,"kb_used_data":498916,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93530316,"statfs":{"total":96636764160,"available":95775043584,"internally_reserved":0,"allocated":510889984,"data_stored":510618602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509595648,"data_stored":509559518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311861248,"data_stored":311777690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":431030272,"data_stored":430990070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:29.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:30.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:30.067 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:30.094 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-11T01:19:28.515456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":581219081,"num_objects":141911,"num_object_clones":0,"num_object_copies":283822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41141,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":141911,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":143334,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6253,"num_bytes_recovered":24773110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2142868,"kb_used_data":1214876,"kb_used_omap":12,"kb_used_meta":927923,"kb_avail":280972652,"statfs":{"total":289910292480,"available":287715995648,"internally_reserved":0,"allocated":1244033024,"data_stored":1243171455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":950193159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,6,5,8,0,0,0,0,0,0,4,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23347200,"num_objects":-5700,"num_object_clones":0,"num_object_copies":-11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-28,"num_objects_misplaced":-291,"num_objects_unfound":0,"num_objects_dirty":-5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5700,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1191936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006105"},"pg_stats":[{"pgid":"2.f","version":"41'11450","reported_seq":11579,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.826662+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:26.826662+0000","last_peered":"2024-09-11T01:19:26.826662+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:26.826662+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11450,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4695},{"shards":"0,2","objects":2792}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11595","reported_seq":11671,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:25.467776+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:25.467776+0000","last_peered":"2024-09-11T01:19:25.467776+0000","last_clean":"2024-09-11T01:19:25.467776+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:25.467776+0000","last_undegraded":"2024-09-11T01:19:25.467776+0000","last_fullsized":"2024-09-11T01:19:25.467776+0000","mapping_epoch":31,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36565958,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9693,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11298","reported_seq":11420,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.341747+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:25.341747+0000","last_peered":"2024-09-11T01:19:25.341747+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:25.341747+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827736,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9359,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2731},{"shards":"2","objects":4764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11482","reported_seq":11523,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:25.360826+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:25.360826+0000","last_peered":"2024-09-11T01:19:25.360826+0000","last_clean":"2024-09-11T01:19:25.360826+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:25.360826+0000","last_undegraded":"2024-09-11T01:19:25.360826+0000","last_fullsized":"2024-09-11T01:19:25.360826+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372480,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11482,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11469","reported_seq":11586,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.851279+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:26.851279+0000","last_peered":"2024-09-11T01:19:26.851279+0000","last_clean":"2024-09-11T01:19:26.851279+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:26.851279+0000","last_undegraded":"2024-09-11T01:19:26.851279+0000","last_fullsized":"2024-09-11T01:19:26.851279+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36078051,"num_objects":8809,"num_object_clones":0,"num_object_copies":17618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8267,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11429","reported_seq":11605,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.900194+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:26.900194+0000","last_peered":"2024-09-11T01:19:26.900194+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:26.900194+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35955171,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7917,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4687},{"shards":"0,2","objects":2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11804","reported_seq":11969,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.913059+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:26.913059+0000","last_peered":"2024-09-11T01:19:26.913059+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:26.913059+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36897734,"num_objects":9010,"num_object_clones":0,"num_object_copies":18020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9010,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8557,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":2888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11568","reported_seq":11683,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.899339+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:26.899339+0000","last_peered":"2024-09-11T01:19:26.899339+0000","last_clean":"2024-09-11T01:19:26.899339+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:26.899339+0000","last_undegraded":"2024-09-11T01:19:26.899339+0000","last_fullsized":"2024-09-11T01:19:26.899339+0000","mapping_epoch":31,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8521,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11428","reported_seq":19766,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:24.349423+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:24.349423+0000","last_peered":"2024-09-11T01:19:24.349423+0000","last_clean":"2024-09-11T01:19:24.349423+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:24.349423+0000","last_undegraded":"2024-09-11T01:19:24.349423+0000","last_fullsized":"2024-09-11T01:19:24.349423+0000","mapping_epoch":40,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36131299,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9295,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11621","reported_seq":11788,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.891280+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:26.891280+0000","last_peered":"2024-09-11T01:19:26.891280+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:26.891280+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7985,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4889},{"shards":"0,2","objects":2785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11365","reported_seq":11477,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.432928+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:25.432928+0000","last_peered":"2024-09-11T01:19:25.432928+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:25.432928+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35938787,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2734},{"shards":"2","objects":4759}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892388+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892388+0000","last_peered":"2024-09-11T01:17:37.892388+0000","last_clean":"2024-09-11T01:17:37.892388+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892388+0000","last_undegraded":"2024-09-11T01:17:37.892388+0000","last_fullsized":"2024-09-11T01:17:37.892388+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11582","reported_seq":11701,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.480913+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:25.480913+0000","last_peered":"2024-09-11T01:19:25.480913+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:25.480913+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2879},{"shards":"2","objects":4818}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11477","reported_seq":15452,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:26.936382+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:26.936382+0000","last_peered":"2024-09-11T01:19:26.936382+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:26.936382+0000","last_undegraded":"2024-09-11T01:19:26.936382+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36352000,"num_objects":8875,"num_object_clones":0,"num_object_copies":17750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":8875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8167,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":7880704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6635},{"shards":"0,2","objects":2240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892517+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892517+0000","last_peered":"2024-09-11T01:17:37.892517+0000","last_clean":"2024-09-11T01:17:37.892517+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892517+0000","last_undegraded":"2024-09-11T01:17:37.892517+0000","last_fullsized":"2024-09-11T01:17:37.892517+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11669","reported_seq":11847,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.858367+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:26.858367+0000","last_peered":"2024-09-11T01:19:26.858367+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:26.858367+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36769792,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4809},{"shards":"0,2","objects":2850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11612","reported_seq":11785,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.861118+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:26.861118+0000","last_peered":"2024-09-11T01:19:26.861118+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:26.861118+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36814848,"num_objects":8988,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892273+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892273+0000","last_peered":"2024-09-11T01:17:37.892273+0000","last_clean":"2024-09-11T01:17:37.892273+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892273+0000","last_undegraded":"2024-09-11T01:17:37.892273+0000","last_fullsized":"2024-09-11T01:17:37.892273+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11420","reported_seq":11579,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.864635+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:26.864635+0000","last_peered":"2024-09-11T01:19:26.864635+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:26.864635+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35709894,"num_objects":8720,"num_object_clones":0,"num_object_copies":17440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8077,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4580},{"shards":"0,2","objects":2809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":581219062,"num_objects":141909,"num_object_clones":0,"num_object_copies":283818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41141,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":141909,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":143332,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6252,"num_bytes_recovered":24773091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1241440256,"data_stored":1241280366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709952,"kb_used_data":417260,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93661888,"statfs":{"total":96636764160,"available":95909773312,"internally_reserved":0,"allocated":427274240,"data_stored":427002863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593556,"kb_used_data":305028,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93778284,"statfs":{"total":96636764160,"available":96028962816,"internally_reserved":0,"allocated":312348672,"data_stored":312029862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295431855},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839360,"kb_used_data":492588,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93532480,"statfs":{"total":96636764160,"available":95777259520,"internally_reserved":0,"allocated":504410112,"data_stored":504138730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,5,0,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":503099392,"data_stored":503063262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311861248,"data_stored":311777690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426479616,"data_stored":426439414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:30.096 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:30.689 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:30.691 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:30.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-11T01:19:30.516288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580752137,"num_objects":141797,"num_object_clones":0,"num_object_copies":283594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41141,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":141797,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":143448,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6253,"num_bytes_recovered":24773110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146388,"kb_used_data":1214236,"kb_used_omap":12,"kb_used_meta":932083,"kb_avail":280969132,"statfs":{"total":289910292480,"available":287712391168,"internally_reserved":0,"allocated":1243377664,"data_stored":1242516095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":954452999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,6,4,1,7,0,0,0,0,0,0,4,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18173952,"num_objects":-4437,"num_object_clones":0,"num_object_copies":-8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-16,"num_objects_misplaced":-208,"num_objects_unfound":0,"num_objects_dirty":-4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006215"},"pg_stats":[{"pgid":"2.f","version":"41'11450","reported_seq":11579,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.826662+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:26.826662+0000","last_peered":"2024-09-11T01:19:26.826662+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:26.826662+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11450,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4695},{"shards":"0,2","objects":2792}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11595","reported_seq":11671,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:25.467776+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:25.467776+0000","last_peered":"2024-09-11T01:19:25.467776+0000","last_clean":"2024-09-11T01:19:25.467776+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:25.467776+0000","last_undegraded":"2024-09-11T01:19:25.467776+0000","last_fullsized":"2024-09-11T01:19:25.467776+0000","mapping_epoch":31,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36565958,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9693,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11298","reported_seq":11420,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.341747+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:25.341747+0000","last_peered":"2024-09-11T01:19:25.341747+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:25.341747+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827736,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9359,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2731},{"shards":"2","objects":4764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11482","reported_seq":11523,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:25.360826+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:25.360826+0000","last_peered":"2024-09-11T01:19:25.360826+0000","last_clean":"2024-09-11T01:19:25.360826+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:25.360826+0000","last_undegraded":"2024-09-11T01:19:25.360826+0000","last_fullsized":"2024-09-11T01:19:25.360826+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372480,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11482,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11469","reported_seq":11586,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.851279+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:26.851279+0000","last_peered":"2024-09-11T01:19:26.851279+0000","last_clean":"2024-09-11T01:19:26.851279+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:26.851279+0000","last_undegraded":"2024-09-11T01:19:26.851279+0000","last_fullsized":"2024-09-11T01:19:26.851279+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36078051,"num_objects":8809,"num_object_clones":0,"num_object_copies":17618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8267,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11429","reported_seq":11605,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.900194+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:26.900194+0000","last_peered":"2024-09-11T01:19:26.900194+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:26.900194+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35955171,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7917,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4687},{"shards":"0,2","objects":2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11804","reported_seq":11969,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.913059+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:26.913059+0000","last_peered":"2024-09-11T01:19:26.913059+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:26.913059+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36897734,"num_objects":9010,"num_object_clones":0,"num_object_copies":18020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9010,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8557,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":2888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11568","reported_seq":11683,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.899339+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:26.899339+0000","last_peered":"2024-09-11T01:19:26.899339+0000","last_clean":"2024-09-11T01:19:26.899339+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:26.899339+0000","last_undegraded":"2024-09-11T01:19:26.899339+0000","last_fullsized":"2024-09-11T01:19:26.899339+0000","mapping_epoch":31,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8521,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11542","reported_seq":19880,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.346316+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:29.346316+0000","last_peered":"2024-09-11T01:19:29.346316+0000","last_clean":"2024-09-11T01:19:29.346316+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:29.346316+0000","last_undegraded":"2024-09-11T01:19:29.346316+0000","last_fullsized":"2024-09-11T01:19:29.346316+0000","mapping_epoch":40,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35664355,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9409,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11621","reported_seq":11788,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.891280+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:26.891280+0000","last_peered":"2024-09-11T01:19:26.891280+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:26.891280+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7985,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4889},{"shards":"0,2","objects":2785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11365","reported_seq":11477,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.432928+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:25.432928+0000","last_peered":"2024-09-11T01:19:25.432928+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:25.432928+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35938787,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2734},{"shards":"2","objects":4759}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11582","reported_seq":11701,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.480913+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:25.480913+0000","last_peered":"2024-09-11T01:19:25.480913+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:25.480913+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2879},{"shards":"2","objects":4818}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11477","reported_seq":15452,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:26.936382+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:26.936382+0000","last_peered":"2024-09-11T01:19:26.936382+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:26.936382+0000","last_undegraded":"2024-09-11T01:19:26.936382+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36352000,"num_objects":8875,"num_object_clones":0,"num_object_copies":17750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":8875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8167,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":7880704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6635},{"shards":"0,2","objects":2240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11669","reported_seq":11847,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.858367+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:26.858367+0000","last_peered":"2024-09-11T01:19:26.858367+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:26.858367+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36769792,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4809},{"shards":"0,2","objects":2850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11612","reported_seq":11785,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.861118+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:26.861118+0000","last_peered":"2024-09-11T01:19:26.861118+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:26.861118+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36814848,"num_objects":8988,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11420","reported_seq":11579,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.864635+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:26.864635+0000","last_peered":"2024-09-11T01:19:26.864635+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:26.864635+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35709894,"num_objects":8720,"num_object_clones":0,"num_object_copies":17440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8077,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4580},{"shards":"0,2","objects":2809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":580752118,"num_objects":141795,"num_object_clones":0,"num_object_copies":283590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41141,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":141795,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":143446,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6252,"num_bytes_recovered":24773091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1240801280,"data_stored":1240641390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709952,"kb_used_data":417260,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93661888,"statfs":{"total":96636764160,"available":95909773312,"internally_reserved":0,"allocated":427274240,"data_stored":427002863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790003,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597076,"kb_used_data":304388,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93774764,"statfs":{"total":96636764160,"available":96025358336,"internally_reserved":0,"allocated":311693312,"data_stored":311374502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839360,"kb_used_data":492588,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93532480,"statfs":{"total":96636764160,"available":95777259520,"internally_reserved":0,"allocated":504410112,"data_stored":504138730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,5,0,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":503099392,"data_stored":503063262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311222272,"data_stored":311138714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426479616,"data_stored":426439414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:30.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:30.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11450", 'reported_seq': 11579, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.826662+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:26.826662+0000', 'last_peered': '2024-09-11T01:19:26.826662+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:26.826662+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35995648, 'num_objects': 8788, 'num_object_clones': 0, 'num_object_copies': 17576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4093, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11450, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4695}, {'shards': '0,2', 'objects': 2792}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:30.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11298", 'reported_seq': 11420, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:25.341747+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:25.341747+0000', 'last_peered': '2024-09-11T01:19:25.341747+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:25.341747+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1200", 'ondisk_log_start': "32'1200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35827736, 'num_objects': 8748, 'num_object_clones': 0, 'num_object_copies': 17496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8748, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9359, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2731}, {'shards': '2', 'objects': 4764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:30.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:30.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11429", 'reported_seq': 11605, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.900194+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:26.900194+0000', 'last_peered': '2024-09-11T01:19:26.900194+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:26.900194+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35955171, 'num_objects': 8779, 'num_object_clones': 0, 'num_object_copies': 17558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4093, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8779, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7917, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4687}, {'shards': '0,2', 'objects': 2791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:30.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11804", 'reported_seq': 11969, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.913059+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:26.913059+0000', 'last_peered': '2024-09-11T01:19:26.913059+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:26.913059+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36897734, 'num_objects': 9010, 'num_object_clones': 0, 'num_object_copies': 18020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4262, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9010, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8557, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4748}, {'shards': '0,2', 'objects': 2888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:30.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'11621", 'reported_seq': 11788, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.891280+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:26.891280+0000', 'last_peered': '2024-09-11T01:19:26.891280+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:26.891280+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'1600", 'ondisk_log_start': "32'1600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36753408, 'num_objects': 8973, 'num_object_clones': 0, 'num_object_copies': 17946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4086, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7985, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4889}, {'shards': '0,2', 'objects': 2785}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'11365", 'reported_seq': 11477, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:25.432928+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:25.432928+0000', 'last_peered': '2024-09-11T01:19:25.432928+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:25.432928+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'1300", 'ondisk_log_start': "32'1300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35938787, 'num_objects': 8775, 'num_object_clones': 0, 'num_object_copies': 17550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4017, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8775, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9134, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 11, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2734}, {'shards': '2', 'objects': 4759}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11582", 'reported_seq': 11701, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:25.480913+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:25.480913+0000', 'last_peered': '2024-09-11T01:19:25.480913+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:25.480913+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36782080, 'num_objects': 8980, 'num_object_clones': 0, 'num_object_copies': 17960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4164, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8980, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9219, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2879}, {'shards': '2', 'objects': 4818}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11477", 'reported_seq': 15452, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:26.936382+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:26.936382+0000', 'last_peered': '2024-09-11T01:19:26.936382+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:26.936382+0000', 'last_undegraded': '2024-09-11T01:19:26.936382+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36352000, 'num_objects': 8875, 'num_object_clones': 0, 'num_object_copies': 17750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2240, 'num_objects_unfound': 0, 'num_objects_dirty': 8875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8167, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1924, 'num_bytes_recovered': 7880704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6635}, {'shards': '0,2', 'objects': 2240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:30.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11669", 'reported_seq': 11847, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.858367+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:26.858367+0000', 'last_peered': '2024-09-11T01:19:26.858367+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:26.858367+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1600", 'ondisk_log_start': "32'1600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36769792, 'num_objects': 8977, 'num_object_clones': 0, 'num_object_copies': 17954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4171, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8062, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4809}, {'shards': '0,2', 'objects': 2850}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:30.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'11612", 'reported_seq': 11785, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.861118+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:26.861118+0000', 'last_peered': '2024-09-11T01:19:26.861118+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:26.861118+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'1600", 'ondisk_log_start': "32'1600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36814848, 'num_objects': 8988, 'num_object_clones': 0, 'num_object_copies': 17976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4128, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8147, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4862}, {'shards': '0,2', 'objects': 2838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:30.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'11420", 'reported_seq': 11579, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:26.864635+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:26.864635+0000', 'last_peered': '2024-09-11T01:19:26.864635+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:26.864635+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35709894, 'num_objects': 8720, 'num_object_clones': 0, 'num_object_copies': 17440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8720, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8077, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 14, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4580}, {'shards': '0,2', 'objects': 2809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:30.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:31.434 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:31.434 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41141/283594 objects degraded (14.507%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":371,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":141797,"data_bytes":580752137,"bytes_used":2197901312,"bytes_avail":287712391168,"bytes_total":289910292480,"degraded_objects":41141,"degraded_total":283594,"degraded_ratio":0.14507006495200886,"misplaced_objects":2240,"misplaced_total":283594,"misplaced_ratio":0.007898615626564737,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":70960,"recovering_keys_per_sec":0,"num_objects_recovered":224,"num_bytes_recovered":851968,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:31.458 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:31.458 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:32.005 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:32.006 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:32.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-11T01:19:30.516288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580752137,"num_objects":141797,"num_object_clones":0,"num_object_copies":283594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41141,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":141797,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":143448,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6253,"num_bytes_recovered":24773110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160886,"ondisk_log_size":160886,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2146388,"kb_used_data":1214236,"kb_used_omap":12,"kb_used_meta":932083,"kb_avail":280969132,"statfs":{"total":289910292480,"available":287712391168,"internally_reserved":0,"allocated":1243377664,"data_stored":1242516095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":954452999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,6,4,1,7,0,0,0,0,0,0,4,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18173952,"num_objects":-4437,"num_object_clones":0,"num_object_copies":-8874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-16,"num_objects_misplaced":-208,"num_objects_unfound":0,"num_objects_dirty":-4437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4437,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":851968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006215"},"pg_stats":[{"pgid":"2.f","version":"41'11450","reported_seq":11579,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.826662+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:26.826662+0000","last_peered":"2024-09-11T01:19:26.826662+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:26.826662+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35995648,"num_objects":8788,"num_object_clones":0,"num_object_copies":17576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11450,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4695},{"shards":"0,2","objects":2792}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11595","reported_seq":11671,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:25.467776+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:25.467776+0000","last_peered":"2024-09-11T01:19:25.467776+0000","last_clean":"2024-09-11T01:19:25.467776+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:25.467776+0000","last_undegraded":"2024-09-11T01:19:25.467776+0000","last_fullsized":"2024-09-11T01:19:25.467776+0000","mapping_epoch":31,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36565958,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9693,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11298","reported_seq":11420,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.341747+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:25.341747+0000","last_peered":"2024-09-11T01:19:25.341747+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:25.341747+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1200","ondisk_log_start":"32'1200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35827736,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9359,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2731},{"shards":"2","objects":4764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11482","reported_seq":11523,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:25.360826+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:25.360826+0000","last_peered":"2024-09-11T01:19:25.360826+0000","last_clean":"2024-09-11T01:19:25.360826+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:25.360826+0000","last_undegraded":"2024-09-11T01:19:25.360826+0000","last_fullsized":"2024-09-11T01:19:25.360826+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372480,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11482,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11469","reported_seq":11586,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.851279+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:26.851279+0000","last_peered":"2024-09-11T01:19:26.851279+0000","last_clean":"2024-09-11T01:19:26.851279+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:26.851279+0000","last_undegraded":"2024-09-11T01:19:26.851279+0000","last_fullsized":"2024-09-11T01:19:26.851279+0000","mapping_epoch":31,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36078051,"num_objects":8809,"num_object_clones":0,"num_object_copies":17618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8809,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8267,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11429","reported_seq":11605,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.900194+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:26.900194+0000","last_peered":"2024-09-11T01:19:26.900194+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:26.900194+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35955171,"num_objects":8779,"num_object_clones":0,"num_object_copies":17558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7917,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4687},{"shards":"0,2","objects":2791}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11804","reported_seq":11969,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.913059+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:26.913059+0000","last_peered":"2024-09-11T01:19:26.913059+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:26.913059+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36897734,"num_objects":9010,"num_object_clones":0,"num_object_copies":18020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9010,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8557,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":2888}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11568","reported_seq":11683,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.899339+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:26.899339+0000","last_peered":"2024-09-11T01:19:26.899339+0000","last_clean":"2024-09-11T01:19:26.899339+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:26.899339+0000","last_undegraded":"2024-09-11T01:19:26.899339+0000","last_fullsized":"2024-09-11T01:19:26.899339+0000","mapping_epoch":31,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8521,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11542","reported_seq":19880,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.346316+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:29.346316+0000","last_peered":"2024-09-11T01:19:29.346316+0000","last_clean":"2024-09-11T01:19:29.346316+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:29.346316+0000","last_undegraded":"2024-09-11T01:19:29.346316+0000","last_fullsized":"2024-09-11T01:19:29.346316+0000","mapping_epoch":40,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35664355,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9409,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11621","reported_seq":11788,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.891280+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:26.891280+0000","last_peered":"2024-09-11T01:19:26.891280+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:26.891280+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7985,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4889},{"shards":"0,2","objects":2785}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11365","reported_seq":11477,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.432928+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:25.432928+0000","last_peered":"2024-09-11T01:19:25.432928+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:25.432928+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1300","ondisk_log_start":"32'1300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35938787,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4017,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2734},{"shards":"2","objects":4759}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11582","reported_seq":11701,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:25.480913+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:25.480913+0000","last_peered":"2024-09-11T01:19:25.480913+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:25.480913+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36782080,"num_objects":8980,"num_object_clones":0,"num_object_copies":17960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8980,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9219,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2879},{"shards":"2","objects":4818}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11477","reported_seq":15452,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:26.936382+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:26.936382+0000","last_peered":"2024-09-11T01:19:26.936382+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:26.936382+0000","last_undegraded":"2024-09-11T01:19:26.936382+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36352000,"num_objects":8875,"num_object_clones":0,"num_object_copies":17750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":8875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8167,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1924,"num_bytes_recovered":7880704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6635},{"shards":"0,2","objects":2240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11669","reported_seq":11847,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.858367+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:26.858367+0000","last_peered":"2024-09-11T01:19:26.858367+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:26.858367+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36769792,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8062,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4809},{"shards":"0,2","objects":2850}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11612","reported_seq":11785,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.861118+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:26.861118+0000","last_peered":"2024-09-11T01:19:26.861118+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:26.861118+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36814848,"num_objects":8988,"num_object_clones":0,"num_object_copies":17976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8147,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":2838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11420","reported_seq":11579,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:26.864635+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:26.864635+0000","last_peered":"2024-09-11T01:19:26.864635+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:26.864635+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35709894,"num_objects":8720,"num_object_clones":0,"num_object_copies":17440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8077,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4580},{"shards":"0,2","objects":2809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":580752118,"num_objects":141795,"num_object_clones":0,"num_object_copies":283590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41141,"num_objects_misplaced":2240,"num_objects_unfound":0,"num_objects_dirty":141795,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":143446,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6252,"num_bytes_recovered":24773091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1240801280,"data_stored":1240641390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160883,"ondisk_log_size":160883,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709952,"kb_used_data":417260,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93661888,"statfs":{"total":96636764160,"available":95909773312,"internally_reserved":0,"allocated":427274240,"data_stored":427002863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,2,0,0,0,0,0,0,0,0,1],"upper_bound":65536},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790003,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597076,"kb_used_data":304388,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93774764,"statfs":{"total":96636764160,"available":96025358336,"internally_reserved":0,"allocated":311693312,"data_stored":311374502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839360,"kb_used_data":492588,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93532480,"statfs":{"total":96636764160,"available":95777259520,"internally_reserved":0,"allocated":504410112,"data_stored":504138730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,5,0,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":503099392,"data_stored":503063262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311222272,"data_stored":311138714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":426479616,"data_stored":426439414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:35.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:35.605 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:35.605 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:35.629 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-11T01:19:34.518131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":571777801,"num_objects":139606,"num_object_clones":0,"num_object_copies":279212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":139606,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":145639,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6379,"num_bytes_recovered":25215478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160577,"ondisk_log_size":160577,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119860,"kb_used_data":1202940,"kb_used_omap":12,"kb_used_meta":916851,"kb_avail":280995660,"statfs":{"total":289910292480,"available":287739555840,"internally_reserved":0,"allocated":1231810560,"data_stored":1230948991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":938855431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,9,9,3,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18374656,"num_objects":-4486,"num_object_clones":0,"num_object_copies":-8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-27,"num_objects_misplaced":-209,"num_objects_unfound":0,"num_objects_dirty":-4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4486,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":856064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005430"},"pg_stats":[{"pgid":"2.f","version":"41'11592","reported_seq":11723,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.915717+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:31.915717+0000","last_peered":"2024-09-11T01:19:31.915717+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:31.915717+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11592,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4554},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11727","reported_seq":11803,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:30.421599+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:30.421599+0000","last_peered":"2024-09-11T01:19:30.421599+0000","last_clean":"2024-09-11T01:19:30.421599+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:30.421599+0000","last_undegraded":"2024-09-11T01:19:30.421599+0000","last_fullsized":"2024-09-11T01:19:30.421599+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36025286,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9825,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11412","reported_seq":11534,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:30.452487+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:30.452487+0000","last_peered":"2024-09-11T01:19:30.452487+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:30.452487+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360792,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9473,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2617},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11630","reported_seq":11671,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:30.450339+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:30.450339+0000","last_peered":"2024-09-11T01:19:30.450339+0000","last_clean":"2024-09-11T01:19:30.450339+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:30.450339+0000","last_undegraded":"2024-09-11T01:19:30.450339+0000","last_fullsized":"2024-09-11T01:19:30.450339+0000","mapping_epoch":31,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11630,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11628","reported_seq":11745,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.932622+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:31.932622+0000","last_peered":"2024-09-11T01:19:31.932622+0000","last_clean":"2024-09-11T01:19:31.932622+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:31.932622+0000","last_undegraded":"2024-09-11T01:19:31.932622+0000","last_fullsized":"2024-09-11T01:19:31.932622+0000","mapping_epoch":31,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426787,"num_objects":8650,"num_object_clones":0,"num_object_copies":17300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8426,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11565","reported_seq":11740,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.923582+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:31.923582+0000","last_peered":"2024-09-11T01:19:31.923582+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:31.923582+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35398115,"num_objects":8643,"num_object_clones":0,"num_object_copies":17286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8053,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4551},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11927","reported_seq":12106,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.860131+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:31.860131+0000","last_peered":"2024-09-11T01:19:31.860131+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:31.860131+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36393926,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8680,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11706","reported_seq":11821,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.831401+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:31.831401+0000","last_peered":"2024-09-11T01:19:31.831401+0000","last_clean":"2024-09-11T01:19:31.831401+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:31.831401+0000","last_undegraded":"2024-09-11T01:19:31.831401+0000","last_fullsized":"2024-09-11T01:19:31.831401+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35708928,"num_objects":8718,"num_object_clones":0,"num_object_copies":17436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8659,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11679","reported_seq":20017,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.268960+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:34.268960+0000","last_peered":"2024-09-11T01:19:34.268960+0000","last_clean":"2024-09-11T01:19:34.268960+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:34.268960+0000","last_undegraded":"2024-09-11T01:19:34.268960+0000","last_fullsized":"2024-09-11T01:19:34.268960+0000","mapping_epoch":40,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35103203,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9546,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11744","reported_seq":11915,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.825332+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:31.825332+0000","last_peered":"2024-09-11T01:19:31.825332+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:31.825332+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4768},{"shards":"0,2","objects":2662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11505","reported_seq":11619,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:30.422930+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:30.422930+0000","last_peered":"2024-09-11T01:19:30.422930+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:30.422930+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35365347,"num_objects":8635,"num_object_clones":0,"num_object_copies":17270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8635,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9274,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2594},{"shards":"2","objects":4620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11719","reported_seq":11844,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:30.484460+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:30.484460+0000","last_peered":"2024-09-11T01:19:30.484460+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:30.484460+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2742},{"shards":"2","objects":4684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11618","reported_seq":15809,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:31.943713+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:31.943713+0000","last_peered":"2024-09-11T01:19:31.943713+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:31.943713+0000","last_undegraded":"2024-09-11T01:19:31.943713+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35774464,"num_objects":8734,"num_object_clones":0,"num_object_copies":17468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":8734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8308,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2032,"num_bytes_recovered":8323072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6602},{"shards":"0,2","objects":2132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11809","reported_seq":11988,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.953967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:31.953967+0000","last_peered":"2024-09-11T01:19:31.953967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:31.953967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36196352,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8202,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4670},{"shards":"0,2","objects":2710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11753","reported_seq":11934,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.909916+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:31.909916+0000","last_peered":"2024-09-11T01:19:31.909916+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:31.909916+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4725},{"shards":"0,2","objects":2697}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11560","reported_seq":11723,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.907579+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:31.907579+0000","last_peered":"2024-09-11T01:19:31.907579+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:31.907579+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35136454,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8217,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2669}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":571777782,"num_objects":139604,"num_object_clones":0,"num_object_copies":279208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":139604,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":145637,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6378,"num_bytes_recovered":25215459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1228951552,"data_stored":1228791662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160574,"ondisk_log_size":160574,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693244,"kb_used_data":413028,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93678596,"statfs":{"total":96636764160,"available":95926882304,"internally_reserved":0,"allocated":422940672,"data_stored":422669295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583968,"kb_used_data":303760,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93787872,"statfs":{"total":96636764160,"available":96038780928,"internally_reserved":0,"allocated":311050240,"data_stored":310731430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842648,"kb_used_data":486152,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93529192,"statfs":{"total":96636764160,"available":95773892608,"internally_reserved":0,"allocated":497819648,"data_stored":497548266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496324608,"data_stored":496288478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310603776,"data_stored":310520218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422023168,"data_stored":421982966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:35.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:36.217 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:36.218 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:36.241 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-11T01:19:34.518131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":571777801,"num_objects":139606,"num_object_clones":0,"num_object_copies":279212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":139606,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":145639,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6379,"num_bytes_recovered":25215478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160577,"ondisk_log_size":160577,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119860,"kb_used_data":1202940,"kb_used_omap":12,"kb_used_meta":916851,"kb_avail":280995660,"statfs":{"total":289910292480,"available":287739555840,"internally_reserved":0,"allocated":1231810560,"data_stored":1230948991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":938855431},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,9,9,3,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18374656,"num_objects":-4486,"num_object_clones":0,"num_object_copies":-8972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-27,"num_objects_misplaced":-209,"num_objects_unfound":0,"num_objects_dirty":-4486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4486,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":856064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005430"},"pg_stats":[{"pgid":"2.f","version":"41'11592","reported_seq":11723,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.915717+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:31.915717+0000","last_peered":"2024-09-11T01:19:31.915717+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:31.915717+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11592,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4554},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11727","reported_seq":11803,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:30.421599+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:30.421599+0000","last_peered":"2024-09-11T01:19:30.421599+0000","last_clean":"2024-09-11T01:19:30.421599+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:30.421599+0000","last_undegraded":"2024-09-11T01:19:30.421599+0000","last_fullsized":"2024-09-11T01:19:30.421599+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36025286,"num_objects":8797,"num_object_clones":0,"num_object_copies":17594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8797,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9825,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11412","reported_seq":11534,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:30.452487+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:30.452487+0000","last_peered":"2024-09-11T01:19:30.452487+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:30.452487+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1400","ondisk_log_start":"32'1400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360792,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9473,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2617},{"shards":"2","objects":4650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11630","reported_seq":11671,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:30.450339+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:30.450339+0000","last_peered":"2024-09-11T01:19:30.450339+0000","last_clean":"2024-09-11T01:19:30.450339+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:30.450339+0000","last_undegraded":"2024-09-11T01:19:30.450339+0000","last_fullsized":"2024-09-11T01:19:30.450339+0000","mapping_epoch":31,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11630,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11628","reported_seq":11745,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.932622+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:31.932622+0000","last_peered":"2024-09-11T01:19:31.932622+0000","last_clean":"2024-09-11T01:19:31.932622+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:31.932622+0000","last_undegraded":"2024-09-11T01:19:31.932622+0000","last_fullsized":"2024-09-11T01:19:31.932622+0000","mapping_epoch":31,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426787,"num_objects":8650,"num_object_clones":0,"num_object_copies":17300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8426,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11565","reported_seq":11740,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.923582+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:31.923582+0000","last_peered":"2024-09-11T01:19:31.923582+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:31.923582+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35398115,"num_objects":8643,"num_object_clones":0,"num_object_copies":17286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8053,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4551},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11927","reported_seq":12106,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.860131+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:31.860131+0000","last_peered":"2024-09-11T01:19:31.860131+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:31.860131+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36393926,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8680,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11706","reported_seq":11821,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.831401+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:31.831401+0000","last_peered":"2024-09-11T01:19:31.831401+0000","last_clean":"2024-09-11T01:19:31.831401+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:31.831401+0000","last_undegraded":"2024-09-11T01:19:31.831401+0000","last_fullsized":"2024-09-11T01:19:31.831401+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35708928,"num_objects":8718,"num_object_clones":0,"num_object_copies":17436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8659,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11679","reported_seq":20017,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.268960+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:34.268960+0000","last_peered":"2024-09-11T01:19:34.268960+0000","last_clean":"2024-09-11T01:19:34.268960+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:34.268960+0000","last_undegraded":"2024-09-11T01:19:34.268960+0000","last_fullsized":"2024-09-11T01:19:34.268960+0000","mapping_epoch":40,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35103203,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9546,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11744","reported_seq":11915,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.825332+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:31.825332+0000","last_peered":"2024-09-11T01:19:31.825332+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:31.825332+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4768},{"shards":"0,2","objects":2662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11505","reported_seq":11619,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:30.422930+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:30.422930+0000","last_peered":"2024-09-11T01:19:30.422930+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:30.422930+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35365347,"num_objects":8635,"num_object_clones":0,"num_object_copies":17270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8635,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9274,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2594},{"shards":"2","objects":4620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11719","reported_seq":11844,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:30.484460+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:30.484460+0000","last_peered":"2024-09-11T01:19:30.484460+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:30.484460+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2742},{"shards":"2","objects":4684}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11618","reported_seq":15809,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:31.943713+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:31.943713+0000","last_peered":"2024-09-11T01:19:31.943713+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:31.943713+0000","last_undegraded":"2024-09-11T01:19:31.943713+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35774464,"num_objects":8734,"num_object_clones":0,"num_object_copies":17468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":8734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8308,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2032,"num_bytes_recovered":8323072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6602},{"shards":"0,2","objects":2132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11809","reported_seq":11988,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.953967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:31.953967+0000","last_peered":"2024-09-11T01:19:31.953967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:31.953967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36196352,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8202,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4670},{"shards":"0,2","objects":2710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11753","reported_seq":11934,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.909916+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:31.909916+0000","last_peered":"2024-09-11T01:19:31.909916+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:31.909916+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4725},{"shards":"0,2","objects":2697}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11560","reported_seq":11723,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.907579+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:31.907579+0000","last_peered":"2024-09-11T01:19:31.907579+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:31.907579+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35136454,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8217,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2669}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":571777782,"num_objects":139604,"num_object_clones":0,"num_object_copies":279208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41123,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":139604,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":145637,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6378,"num_bytes_recovered":25215459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1228951552,"data_stored":1228791662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160574,"ondisk_log_size":160574,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693244,"kb_used_data":413028,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93678596,"statfs":{"total":96636764160,"available":95926882304,"internally_reserved":0,"allocated":422940672,"data_stored":422669295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583968,"kb_used_data":303760,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93787872,"statfs":{"total":96636764160,"available":96038780928,"internally_reserved":0,"allocated":311050240,"data_stored":310731430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842648,"kb_used_data":486152,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93529192,"statfs":{"total":96636764160,"available":95773892608,"internally_reserved":0,"allocated":497819648,"data_stored":497548266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496324608,"data_stored":496288478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310603776,"data_stored":310520218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":422023168,"data_stored":421982966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:36.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:36.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11592", 'reported_seq': 11723, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.915717+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:31.915717+0000', 'last_peered': '2024-09-11T01:19:31.915717+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:31.915717+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35414016, 'num_objects': 8646, 'num_object_clones': 0, 'num_object_copies': 17292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8646, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11592, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4554}, {'shards': '0,2', 'objects': 2650}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:36.242 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11412", 'reported_seq': 11534, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:30.452487+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:30.452487+0000', 'last_peered': '2024-09-11T01:19:30.452487+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:30.452487+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1400", 'ondisk_log_start': "32'1400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35360792, 'num_objects': 8634, 'num_object_clones': 0, 'num_object_copies': 17268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8634, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9473, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2617}, {'shards': '2', 'objects': 4650}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:36.242 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11565", 'reported_seq': 11740, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.923582+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:31.923582+0000', 'last_peered': '2024-09-11T01:19:31.923582+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:31.923582+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35398115, 'num_objects': 8643, 'num_object_clones': 0, 'num_object_copies': 17286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4093, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8643, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8053, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4551}, {'shards': '0,2', 'objects': 2655}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'11927", 'reported_seq': 12106, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.860131+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:31.860131+0000', 'last_peered': '2024-09-11T01:19:31.860131+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:31.860131+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'1900", 'ondisk_log_start': "32'1900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36393926, 'num_objects': 8887, 'num_object_clones': 0, 'num_object_copies': 17774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4258, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8887, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8680, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4629}, {'shards': '0,2', 'objects': 2765}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'11744", 'reported_seq': 11915, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.825332+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:31.825332+0000', 'last_peered': '2024-09-11T01:19:31.825332+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:31.825332+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36249600, 'num_objects': 8850, 'num_object_clones': 0, 'num_object_copies': 17700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8108, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4768}, {'shards': '0,2', 'objects': 2662}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'11505", 'reported_seq': 11619, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:30.422930+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:30.422930+0000', 'last_peered': '2024-09-11T01:19:30.422930+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:30.422930+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35365347, 'num_objects': 8635, 'num_object_clones': 0, 'num_object_copies': 17270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4016, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8635, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9274, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 12, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2594}, {'shards': '2', 'objects': 4620}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11719", 'reported_seq': 11844, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:30.484460+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:30.484460+0000', 'last_peered': '2024-09-11T01:19:30.484460+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:30.484460+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36220928, 'num_objects': 8843, 'num_object_clones': 0, 'num_object_copies': 17686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8843, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9356, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 18, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2742}, {'shards': '2', 'objects': 4684}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:36.243 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11618", 'reported_seq': 15809, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:31.943713+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:31.943713+0000', 'last_peered': '2024-09-11T01:19:31.943713+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:31.943713+0000', 'last_undegraded': '2024-09-11T01:19:31.943713+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1600", 'ondisk_log_start': "32'1600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35774464, 'num_objects': 8734, 'num_object_clones': 0, 'num_object_copies': 17468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2132, 'num_objects_unfound': 0, 'num_objects_dirty': 8734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8308, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2032, 'num_bytes_recovered': 8323072, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6602}, {'shards': '0,2', 'objects': 2132}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:36.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11809", 'reported_seq': 11988, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.953967+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:31.953967+0000', 'last_peered': '2024-09-11T01:19:31.953967+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:31.953967+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36196352, 'num_objects': 8837, 'num_object_clones': 0, 'num_object_copies': 17674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4170, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8202, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4670}, {'shards': '0,2', 'objects': 2710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:36.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'11753", 'reported_seq': 11934, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.909916+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:31.909916+0000', 'last_peered': '2024-09-11T01:19:31.909916+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:31.909916+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36237312, 'num_objects': 8847, 'num_object_clones': 0, 'num_object_copies': 17694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8847, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8288, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4725}, {'shards': '0,2', 'objects': 2697}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:36.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'11560", 'reported_seq': 11723, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:31.907579+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:31.907579+0000', 'last_peered': '2024-09-11T01:19:31.907579+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:31.907579+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35136454, 'num_objects': 8580, 'num_object_clones': 0, 'num_object_copies': 17160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4140, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8580, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8217, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4442}, {'shards': '0,2', 'objects': 2669}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:36.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:36.995 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:36.995 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41118/277852 objects degraded (14.799%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":377,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138926,"data_bytes":568992521,"bytes_used":2170810368,"bytes_avail":287739482112,"bytes_total":289910292480,"degraded_objects":41118,"degraded_total":277852,"degraded_ratio":0.14798525833897183,"misplaced_objects":2132,"misplaced_total":277852,"misplaced_ratio":0.0076731497343909705,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":71286,"recovering_keys_per_sec":0,"num_objects_recovered":241,"num_bytes_recovered":856064,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":418},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:37.019 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:37.020 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:37.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:37.598 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:37.628 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-11T01:19:36.522504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568992521,"num_objects":138926,"num_object_clones":0,"num_object_copies":277852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41118,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":138926,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":146319,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6384,"num_bytes_recovered":25215478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2119932,"kb_used_data":1198860,"kb_used_omap":12,"kb_used_meta":921011,"kb_avail":280995588,"statfs":{"total":289910292480,"available":287739482112,"internally_reserved":0,"allocated":1227632640,"data_stored":1226771071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":943115271},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,7,8,3,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20561920,"num_objects":-5020,"num_object_clones":0,"num_object_copies":-10040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":-209,"num_objects_unfound":0,"num_objects_dirty":-5020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5020,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":856064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008845"},"pg_stats":[{"pgid":"2.f","version":"41'11592","reported_seq":11723,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.915717+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:31.915717+0000","last_peered":"2024-09-11T01:19:31.915717+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:31.915717+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35414016,"num_objects":8646,"num_object_clones":0,"num_object_copies":17292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11592,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4554},{"shards":"0,2","objects":2650}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11861","reported_seq":11937,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:35.488298+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:35.488298+0000","last_peered":"2024-09-11T01:19:35.488298+0000","last_clean":"2024-09-11T01:19:35.488298+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:35.488298+0000","last_undegraded":"2024-09-11T01:19:35.488298+0000","last_fullsized":"2024-09-11T01:19:35.488298+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35476422,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9959,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11547","reported_seq":11669,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.513753+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:35.513753+0000","last_peered":"2024-09-11T01:19:35.513753+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:35.513753+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807832,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9608,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2482},{"shards":"2","objects":4515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11773","reported_seq":11814,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:35.496014+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:35.496014+0000","last_peered":"2024-09-11T01:19:35.496014+0000","last_clean":"2024-09-11T01:19:35.496014+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:35.496014+0000","last_undegraded":"2024-09-11T01:19:35.496014+0000","last_fullsized":"2024-09-11T01:19:35.496014+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11773,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11628","reported_seq":11745,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.932622+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:31.932622+0000","last_peered":"2024-09-11T01:19:31.932622+0000","last_clean":"2024-09-11T01:19:31.932622+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:31.932622+0000","last_undegraded":"2024-09-11T01:19:31.932622+0000","last_fullsized":"2024-09-11T01:19:31.932622+0000","mapping_epoch":31,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426787,"num_objects":8650,"num_object_clones":0,"num_object_copies":17300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8426,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11565","reported_seq":11740,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.923582+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:31.923582+0000","last_peered":"2024-09-11T01:19:31.923582+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:31.923582+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35398115,"num_objects":8643,"num_object_clones":0,"num_object_copies":17286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4093,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8053,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4551},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'11927","reported_seq":12106,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.860131+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:31.860131+0000","last_peered":"2024-09-11T01:19:31.860131+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:31.860131+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36393926,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8680,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4629},{"shards":"0,2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11706","reported_seq":11821,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.831401+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:31.831401+0000","last_peered":"2024-09-11T01:19:31.831401+0000","last_clean":"2024-09-11T01:19:31.831401+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:31.831401+0000","last_undegraded":"2024-09-11T01:19:31.831401+0000","last_fullsized":"2024-09-11T01:19:31.831401+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35708928,"num_objects":8718,"num_object_clones":0,"num_object_copies":17436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8659,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11679","reported_seq":20017,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.268960+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:34.268960+0000","last_peered":"2024-09-11T01:19:34.268960+0000","last_clean":"2024-09-11T01:19:34.268960+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:34.268960+0000","last_undegraded":"2024-09-11T01:19:34.268960+0000","last_fullsized":"2024-09-11T01:19:34.268960+0000","mapping_epoch":40,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35103203,"num_objects":8571,"num_object_clones":0,"num_object_copies":17142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8571,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9546,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11744","reported_seq":11915,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.825332+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:31.825332+0000","last_peered":"2024-09-11T01:19:31.825332+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:31.825332+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36249600,"num_objects":8850,"num_object_clones":0,"num_object_copies":17700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4768},{"shards":"0,2","objects":2662}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11641","reported_seq":11761,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.381119+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:35.381119+0000","last_peered":"2024-09-11T01:19:35.381119+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:35.381119+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34808291,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9410,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2458},{"shards":"2","objects":4487}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11851","reported_seq":11979,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.514831+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:35.514831+0000","last_peered":"2024-09-11T01:19:35.514831+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:35.514831+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35680256,"num_objects":8711,"num_object_clones":0,"num_object_copies":17422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9488,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2610},{"shards":"2","objects":4554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11618","reported_seq":15809,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:31.943713+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:31.943713+0000","last_peered":"2024-09-11T01:19:31.943713+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:31.943713+0000","last_undegraded":"2024-09-11T01:19:31.943713+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35774464,"num_objects":8734,"num_object_clones":0,"num_object_copies":17468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":8734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8308,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2032,"num_bytes_recovered":8323072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6602},{"shards":"0,2","objects":2132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11809","reported_seq":11988,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.953967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:31.953967+0000","last_peered":"2024-09-11T01:19:31.953967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:31.953967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36196352,"num_objects":8837,"num_object_clones":0,"num_object_copies":17674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8202,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4670},{"shards":"0,2","objects":2710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11753","reported_seq":11934,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.909916+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:31.909916+0000","last_peered":"2024-09-11T01:19:31.909916+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:31.909916+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8288,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4725},{"shards":"0,2","objects":2697}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893916+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893916+0000","last_peered":"2024-09-11T01:17:37.893916+0000","last_clean":"2024-09-11T01:17:37.893916+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893916+0000","last_undegraded":"2024-09-11T01:17:37.893916+0000","last_fullsized":"2024-09-11T01:17:37.893916+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11560","reported_seq":11723,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:31.907579+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:31.907579+0000","last_peered":"2024-09-11T01:19:31.907579+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:31.907579+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35136454,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8217,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2669}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568992502,"num_objects":138924,"num_object_clones":0,"num_object_copies":277848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41118,"num_objects_misplaced":2132,"num_objects_unfound":0,"num_objects_dirty":138924,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":146317,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6383,"num_bytes_recovered":25215459,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1224556544,"data_stored":1224396654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693316,"kb_used_data":408948,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93678524,"statfs":{"total":96636764160,"available":95926808576,"internally_reserved":0,"allocated":418762752,"data_stored":418491375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790004,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583968,"kb_used_data":303760,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93787872,"statfs":{"total":96636764160,"available":96038780928,"internally_reserved":0,"allocated":311050240,"data_stored":310731430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":842648,"kb_used_data":486152,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93529192,"statfs":{"total":96636764160,"available":95773892608,"internally_reserved":0,"allocated":497819648,"data_stored":497548266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":496324608,"data_stored":496288478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310603776,"data_stored":310520218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417628160,"data_stored":417587958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:40.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:41.195 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:41.197 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:41.223 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-11T01:19:40.524098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":563028745,"num_objects":137470,"num_object_clones":0,"num_object_copies":274940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41111,"num_objects_misplaced":2009,"num_objects_unfound":0,"num_objects_dirty":137470,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":147775,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6514,"num_bytes_recovered":25719286,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2143524,"kb_used_data":1191924,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280971996,"statfs":{"total":289910292480,"available":287715323904,"internally_reserved":0,"allocated":1220530176,"data_stored":1219668607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":974375943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,16,5,0,0,0,0,2,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18190336,"num_objects":-4441,"num_object_clones":0,"num_object_copies":-8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":-4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4441,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008642"},"pg_stats":[{"pgid":"2.f","version":"41'11723","reported_seq":11854,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.936417+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:36.936417+0000","last_peered":"2024-09-11T01:19:36.936417+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:36.936417+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877440,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11723,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4423},{"shards":"0,2","objects":2519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11861","reported_seq":11937,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:35.488298+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:35.488298+0000","last_peered":"2024-09-11T01:19:35.488298+0000","last_clean":"2024-09-11T01:19:35.488298+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:35.488298+0000","last_undegraded":"2024-09-11T01:19:35.488298+0000","last_fullsized":"2024-09-11T01:19:35.488298+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35476422,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9959,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11547","reported_seq":11669,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.513753+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:35.513753+0000","last_peered":"2024-09-11T01:19:35.513753+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:35.513753+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807832,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9608,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2482},{"shards":"2","objects":4515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11773","reported_seq":11814,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:35.496014+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:35.496014+0000","last_peered":"2024-09-11T01:19:35.496014+0000","last_clean":"2024-09-11T01:19:35.496014+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:35.496014+0000","last_undegraded":"2024-09-11T01:19:35.496014+0000","last_fullsized":"2024-09-11T01:19:35.496014+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11773,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11757","reported_seq":11873,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.954519+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:36.954519+0000","last_peered":"2024-09-11T01:19:36.954519+0000","last_clean":"2024-09-11T01:19:36.954519+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:36.954519+0000","last_undegraded":"2024-09-11T01:19:36.954519+0000","last_fullsized":"2024-09-11T01:19:36.954519+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34898403,"num_objects":8521,"num_object_clones":0,"num_object_copies":17042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8555,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11704","reported_seq":11884,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.862332+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:36.862332+0000","last_peered":"2024-09-11T01:19:36.862332+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:36.862332+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828771,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8192,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4414},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12046","reported_seq":12227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.783578+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:36.783578+0000","last_peered":"2024-09-11T01:19:36.783578+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:36.783578+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35906502,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8799,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4511},{"shards":"0,2","objects":2646}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11831","reported_seq":11946,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.885114+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:36.885114+0000","last_peered":"2024-09-11T01:19:36.885114+0000","last_clean":"2024-09-11T01:19:36.885114+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:36.885114+0000","last_undegraded":"2024-09-11T01:19:36.885114+0000","last_fullsized":"2024-09-11T01:19:36.885114+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8784,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11819","reported_seq":20157,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.423566+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:39.423566+0000","last_peered":"2024-09-11T01:19:39.423566+0000","last_clean":"2024-09-11T01:19:39.423566+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:39.423566+0000","last_undegraded":"2024-09-11T01:19:39.423566+0000","last_fullsized":"2024-09-11T01:19:39.423566+0000","mapping_epoch":40,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529763,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11887","reported_seq":12061,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.928816+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:36.928816+0000","last_peered":"2024-09-11T01:19:36.928816+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:36.928816+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35663872,"num_objects":8707,"num_object_clones":0,"num_object_copies":17414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8251,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":2519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11641","reported_seq":11761,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.381119+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:35.381119+0000","last_peered":"2024-09-11T01:19:35.381119+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:35.381119+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34808291,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9410,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2458},{"shards":"2","objects":4487}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11851","reported_seq":11979,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.514831+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:35.514831+0000","last_peered":"2024-09-11T01:19:35.514831+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:35.514831+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35680256,"num_objects":8711,"num_object_clones":0,"num_object_copies":17422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9488,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2610},{"shards":"2","objects":4554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11748","reported_seq":16185,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:36.989122+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:36.989122+0000","last_peered":"2024-09-11T01:19:36.989122+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:36.989122+0000","last_undegraded":"2024-09-11T01:19:36.989122+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2009,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2155,"num_bytes_recovered":8826880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6595},{"shards":"0,2","objects":2009}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11949","reported_seq":12129,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.952699+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:36.952699+0000","last_peered":"2024-09-11T01:19:36.952699+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:36.952699+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8342,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4530},{"shards":"0,2","objects":2570}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11872","reported_seq":12057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.949695+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:36.949695+0000","last_peered":"2024-09-11T01:19:36.949695+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:36.949695+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35749888,"num_objects":8728,"num_object_clones":0,"num_object_copies":17456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8407,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11701","reported_seq":11864,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.949741+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:36.949741+0000","last_peered":"2024-09-11T01:19:36.949741+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:36.949741+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34558918,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8358,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4301},{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":563028726,"num_objects":137468,"num_object_clones":0,"num_object_copies":274936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41111,"num_objects_misplaced":2009,"num_objects_unfound":0,"num_objects_dirty":137468,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":147773,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6513,"num_bytes_recovered":25719267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1217396736,"data_stored":1217240459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693316,"kb_used_data":408948,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93678524,"statfs":{"total":96636764160,"available":95926808576,"internally_reserved":0,"allocated":418762752,"data_stored":418491375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609732,"kb_used_data":303156,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93762108,"statfs":{"total":96636764160,"available":96012398592,"internally_reserved":0,"allocated":310431744,"data_stored":310112934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840476,"kb_used_data":479820,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93531364,"statfs":{"total":96636764160,"available":95776116736,"internally_reserved":0,"allocated":491335680,"data_stored":491064298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,4,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489799680,"data_stored":489763550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309968896,"data_stored":309888951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417628160,"data_stored":417587958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:41.224 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:41.772 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:41.775 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:41.795 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-11T01:19:40.524098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":563028745,"num_objects":137470,"num_object_clones":0,"num_object_copies":274940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41111,"num_objects_misplaced":2009,"num_objects_unfound":0,"num_objects_dirty":137470,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":147775,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6514,"num_bytes_recovered":25719286,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2143524,"kb_used_data":1191924,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280971996,"statfs":{"total":289910292480,"available":287715323904,"internally_reserved":0,"allocated":1220530176,"data_stored":1219668607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":974375943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,16,5,0,0,0,0,2,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18190336,"num_objects":-4441,"num_object_clones":0,"num_object_copies":-8882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-231,"num_objects_unfound":0,"num_objects_dirty":-4441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4441,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":946176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008642"},"pg_stats":[{"pgid":"2.f","version":"41'11723","reported_seq":11854,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.936417+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:36.936417+0000","last_peered":"2024-09-11T01:19:36.936417+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:36.936417+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34877440,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11723,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4423},{"shards":"0,2","objects":2519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11861","reported_seq":11937,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:35.488298+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:35.488298+0000","last_peered":"2024-09-11T01:19:35.488298+0000","last_clean":"2024-09-11T01:19:35.488298+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:35.488298+0000","last_undegraded":"2024-09-11T01:19:35.488298+0000","last_fullsized":"2024-09-11T01:19:35.488298+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35476422,"num_objects":8663,"num_object_clones":0,"num_object_copies":17326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8663,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9959,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11547","reported_seq":11669,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.513753+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:35.513753+0000","last_peered":"2024-09-11T01:19:35.513753+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:35.513753+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1500","ondisk_log_start":"32'1500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34807832,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9608,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2482},{"shards":"2","objects":4515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11773","reported_seq":11814,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:35.496014+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:35.496014+0000","last_peered":"2024-09-11T01:19:35.496014+0000","last_clean":"2024-09-11T01:19:35.496014+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:35.496014+0000","last_undegraded":"2024-09-11T01:19:35.496014+0000","last_fullsized":"2024-09-11T01:19:35.496014+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11773,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11757","reported_seq":11873,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.954519+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:36.954519+0000","last_peered":"2024-09-11T01:19:36.954519+0000","last_clean":"2024-09-11T01:19:36.954519+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:36.954519+0000","last_undegraded":"2024-09-11T01:19:36.954519+0000","last_fullsized":"2024-09-11T01:19:36.954519+0000","mapping_epoch":31,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34898403,"num_objects":8521,"num_object_clones":0,"num_object_copies":17042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8521,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8555,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11704","reported_seq":11884,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.862332+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:36.862332+0000","last_peered":"2024-09-11T01:19:36.862332+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:36.862332+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34828771,"num_objects":8504,"num_object_clones":0,"num_object_copies":17008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8192,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4414},{"shards":"0,2","objects":2516}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12046","reported_seq":12227,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.783578+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:36.783578+0000","last_peered":"2024-09-11T01:19:36.783578+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:36.783578+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35906502,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8799,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4511},{"shards":"0,2","objects":2646}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11831","reported_seq":11946,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.885114+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:36.885114+0000","last_peered":"2024-09-11T01:19:36.885114+0000","last_clean":"2024-09-11T01:19:36.885114+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:36.885114+0000","last_undegraded":"2024-09-11T01:19:36.885114+0000","last_fullsized":"2024-09-11T01:19:36.885114+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8784,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11819","reported_seq":20157,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.423566+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:39.423566+0000","last_peered":"2024-09-11T01:19:39.423566+0000","last_clean":"2024-09-11T01:19:39.423566+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:39.423566+0000","last_undegraded":"2024-09-11T01:19:39.423566+0000","last_fullsized":"2024-09-11T01:19:39.423566+0000","mapping_epoch":40,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529763,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'11887","reported_seq":12061,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.928816+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:36.928816+0000","last_peered":"2024-09-11T01:19:36.928816+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:36.928816+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35663872,"num_objects":8707,"num_object_clones":0,"num_object_copies":17414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8251,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4627},{"shards":"0,2","objects":2519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11641","reported_seq":11761,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.381119+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:35.381119+0000","last_peered":"2024-09-11T01:19:35.381119+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:35.381119+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34808291,"num_objects":8499,"num_object_clones":0,"num_object_copies":16998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8499,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9410,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2458},{"shards":"2","objects":4487}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11851","reported_seq":11979,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:35.514831+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:35.514831+0000","last_peered":"2024-09-11T01:19:35.514831+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:35.514831+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35680256,"num_objects":8711,"num_object_clones":0,"num_object_copies":17422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9488,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2610},{"shards":"2","objects":4554}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11748","reported_seq":16185,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:36.989122+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:36.989122+0000","last_peered":"2024-09-11T01:19:36.989122+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:36.989122+0000","last_undegraded":"2024-09-11T01:19:36.989122+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35241984,"num_objects":8604,"num_object_clones":0,"num_object_copies":17208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2009,"num_objects_unfound":0,"num_objects_dirty":8604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2155,"num_bytes_recovered":8826880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6595},{"shards":"0,2","objects":2009}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'11949","reported_seq":12129,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.952699+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:36.952699+0000","last_peered":"2024-09-11T01:19:36.952699+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:36.952699+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4170,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8342,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4530},{"shards":"0,2","objects":2570}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'11872","reported_seq":12057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.949695+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:36.949695+0000","last_peered":"2024-09-11T01:19:36.949695+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:36.949695+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35749888,"num_objects":8728,"num_object_clones":0,"num_object_copies":17456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8407,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11701","reported_seq":11864,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:36.949741+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:36.949741+0000","last_peered":"2024-09-11T01:19:36.949741+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:36.949741+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34558918,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8358,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4301},{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":563028726,"num_objects":137468,"num_object_clones":0,"num_object_copies":274936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41111,"num_objects_misplaced":2009,"num_objects_unfound":0,"num_objects_dirty":137468,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":147773,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6513,"num_bytes_recovered":25719267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1217396736,"data_stored":1217240459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693316,"kb_used_data":408948,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93678524,"statfs":{"total":96636764160,"available":95926808576,"internally_reserved":0,"allocated":418762752,"data_stored":418491375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,4,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609732,"kb_used_data":303156,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93762108,"statfs":{"total":96636764160,"available":96012398592,"internally_reserved":0,"allocated":310431744,"data_stored":310112934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840476,"kb_used_data":479820,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93531364,"statfs":{"total":96636764160,"available":95776116736,"internally_reserved":0,"allocated":491335680,"data_stored":491064298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,4,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":489799680,"data_stored":489763550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309968896,"data_stored":309888951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":417628160,"data_stored":417587958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11723", 'reported_seq': 11854, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.936417+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:36.936417+0000', 'last_peered': '2024-09-11T01:19:36.936417+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:36.936417+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34877440, 'num_objects': 8515, 'num_object_clones': 0, 'num_object_copies': 17030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8515, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11723, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4423}, {'shards': '0,2', 'objects': 2519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11547", 'reported_seq': 11669, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:35.513753+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:35.513753+0000', 'last_peered': '2024-09-11T01:19:35.513753+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:35.513753+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1500", 'ondisk_log_start': "32'1500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34807832, 'num_objects': 8499, 'num_object_clones': 0, 'num_object_copies': 16998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8499, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9608, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2482}, {'shards': '2', 'objects': 4515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11704", 'reported_seq': 11884, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.862332+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:36.862332+0000', 'last_peered': '2024-09-11T01:19:36.862332+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:36.862332+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34828771, 'num_objects': 8504, 'num_object_clones': 0, 'num_object_copies': 17008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8504, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8192, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4414}, {'shards': '0,2', 'objects': 2516}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'12046", 'reported_seq': 12227, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.783578+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:36.783578+0000', 'last_peered': '2024-09-11T01:19:36.783578+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:36.783578+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'2000", 'ondisk_log_start': "32'2000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35906502, 'num_objects': 8768, 'num_object_clones': 0, 'num_object_copies': 17536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8768, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8799, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4511}, {'shards': '0,2', 'objects': 2646}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:41.797 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'11887", 'reported_seq': 12061, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.928816+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:36.928816+0000', 'last_peered': '2024-09-11T01:19:36.928816+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:36.928816+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35663872, 'num_objects': 8707, 'num_object_clones': 0, 'num_object_copies': 17414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4082, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8707, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8251, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4627}, {'shards': '0,2', 'objects': 2519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'11641", 'reported_seq': 11761, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:35.381119+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:35.381119+0000', 'last_peered': '2024-09-11T01:19:35.381119+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:35.381119+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'1600", 'ondisk_log_start': "32'1600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34808291, 'num_objects': 8499, 'num_object_clones': 0, 'num_object_copies': 16998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4013, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8499, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9410, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 15, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2458}, {'shards': '2', 'objects': 4487}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'11851", 'reported_seq': 11979, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:35.514831+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:35.514831+0000', 'last_peered': '2024-09-11T01:19:35.514831+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:35.514831+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35680256, 'num_objects': 8711, 'num_object_clones': 0, 'num_object_copies': 17422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4159, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9488, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 20, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2610}, {'shards': '2', 'objects': 4554}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11748", 'reported_seq': 16185, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:36.989122+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:36.989122+0000', 'last_peered': '2024-09-11T01:19:36.989122+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:36.989122+0000', 'last_undegraded': '2024-09-11T01:19:36.989122+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35241984, 'num_objects': 8604, 'num_object_clones': 0, 'num_object_copies': 17208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2009, 'num_objects_unfound': 0, 'num_objects_dirty': 8604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8438, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2155, 'num_bytes_recovered': 8826880, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6595}, {'shards': '0,2', 'objects': 2009}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'11949", 'reported_seq': 12129, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.952699+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:36.952699+0000', 'last_peered': '2024-09-11T01:19:36.952699+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:36.952699+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'1900", 'ondisk_log_start': "32'1900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35622912, 'num_objects': 8697, 'num_object_clones': 0, 'num_object_copies': 17394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4170, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8342, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4530}, {'shards': '0,2', 'objects': 2570}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:41.798 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'11872", 'reported_seq': 12057, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.949695+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:36.949695+0000', 'last_peered': '2024-09-11T01:19:36.949695+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:36.949695+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35749888, 'num_objects': 8728, 'num_object_clones': 0, 'num_object_copies': 17456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8407, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4608}, {'shards': '0,2', 'objects': 2578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.799 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:41.799 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'11701", 'reported_seq': 11864, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:36.949741+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:36.949741+0000', 'last_peered': '2024-09-11T01:19:36.949741+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:36.949741+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'1700", 'ondisk_log_start': "32'1700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34558918, 'num_objects': 8439, 'num_object_clones': 0, 'num_object_copies': 16878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4140, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8439, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8358, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4301}, {'shards': '0,2', 'objects': 2528}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:41.799 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:42.499 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:42.499 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41111/274940 objects degraded (14.953%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":382,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137470,"data_bytes":563028745,"bytes_used":2194968576,"bytes_avail":287715323904,"bytes_total":289910292480,"degraded_objects":41111,"degraded_total":274940,"degraded_ratio":0.14952716956426856,"misplaced_objects":2009,"misplaced_total":274940,"misplaced_ratio":0.0073070488106495961,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":78791,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":946176,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:42.523 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:42.524 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:43.103 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:43.104 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:43.137 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-11T01:19:42.524843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554808073,"num_objects":135463,"num_object_clones":0,"num_object_copies":270926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41102,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":135463,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":149782,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6649,"num_bytes_recovered":26231286,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141216,"kb_used_data":1181296,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280974304,"statfs":{"total":289910292480,"available":287717687296,"internally_reserved":0,"allocated":1209647104,"data_stored":1208785535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":982895623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1,0,6,13,3,1,0,0,4,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25944064,"num_objects":-6334,"num_object_clones":0,"num_object_copies":-12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-356,"num_objects_unfound":0,"num_objects_dirty":-6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6334,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008555"},"pg_stats":[{"pgid":"2.f","version":"41'11850","reported_seq":11989,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.866441+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:41.866441+0000","last_peered":"2024-09-11T01:19:41.866441+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:41.866441+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34357248,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11850,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":2392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'11989","reported_seq":12065,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:40.490014+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:40.490014+0000","last_peered":"2024-09-11T01:19:40.490014+0000","last_clean":"2024-09-11T01:19:40.490014+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:40.490014+0000","last_undegraded":"2024-09-11T01:19:40.490014+0000","last_fullsized":"2024-09-11T01:19:40.490014+0000","mapping_epoch":31,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34952134,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10087,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11675","reported_seq":11798,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:40.495007+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:40.495007+0000","last_peered":"2024-09-11T01:19:40.495007+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:40.495007+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1600","ondisk_log_start":"32'1600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34283544,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9736,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2354},{"shards":"2","objects":4388}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'11906","reported_seq":11947,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:40.490065+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:40.490065+0000","last_peered":"2024-09-11T01:19:40.490065+0000","last_clean":"2024-09-11T01:19:40.490065+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:40.490065+0000","last_undegraded":"2024-09-11T01:19:40.490065+0000","last_fullsized":"2024-09-11T01:19:40.490065+0000","mapping_epoch":31,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34635776,"num_objects":8456,"num_object_clones":0,"num_object_copies":16912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11906,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11898","reported_seq":12015,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:41.784320+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:41.784320+0000","last_peered":"2024-09-11T01:19:41.784320+0000","last_clean":"2024-09-11T01:19:41.784320+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:41.784320+0000","last_undegraded":"2024-09-11T01:19:41.784320+0000","last_fullsized":"2024-09-11T01:19:41.784320+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34320867,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8696,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11850","reported_seq":12030,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.891543+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:41.891543+0000","last_peered":"2024-09-11T01:19:41.891543+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:41.891543+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8338,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4268},{"shards":"0,2","objects":2370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12161","reported_seq":12342,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.899769+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:41.899769+0000","last_peered":"2024-09-11T01:19:41.899769+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:41.899769+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35435462,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8914,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4396},{"shards":"0,2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11951","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.000011+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:42.000011+0000","last_peered":"2024-09-11T01:19:42.000011+0000","last_clean":"2024-09-11T01:19:42.000011+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:42.000011+0000","last_undegraded":"2024-09-11T01:19:42.000011+0000","last_fullsized":"2024-09-11T01:19:42.000011+0000","mapping_epoch":31,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8904,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11819","reported_seq":20157,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.423566+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:39.423566+0000","last_peered":"2024-09-11T01:19:39.423566+0000","last_clean":"2024-09-11T01:19:39.423566+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:39.423566+0000","last_undegraded":"2024-09-11T01:19:39.423566+0000","last_fullsized":"2024-09-11T01:19:39.423566+0000","mapping_epoch":40,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34529763,"num_objects":8431,"num_object_clones":0,"num_object_copies":16862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12004","reported_seq":12179,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.985498+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:41.985498+0000","last_peered":"2024-09-11T01:19:41.985498+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:41.985498+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35184640,"num_objects":8590,"num_object_clones":0,"num_object_copies":17180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8368,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4510},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11788","reported_seq":11914,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:40.471004+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:40.471004+0000","last_peered":"2024-09-11T01:19:40.471004+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:40.471004+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1700","ondisk_log_start":"32'1700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34206179,"num_objects":8352,"num_object_clones":0,"num_object_copies":16704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8352,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9557,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2311},{"shards":"2","objects":4343}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'11984","reported_seq":12113,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:40.476971+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:40.476971+0000","last_peered":"2024-09-11T01:19:40.476971+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:40.476971+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35135488,"num_objects":8578,"num_object_clones":0,"num_object_copies":17156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9621,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2477},{"shards":"2","objects":4421}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11889","reported_seq":16577,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:41.981872+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:41.981872+0000","last_peered":"2024-09-11T01:19:41.981872+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:41.981872+0000","last_undegraded":"2024-09-11T01:19:41.981872+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34664448,"num_objects":8463,"num_object_clones":0,"num_object_copies":16926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":8463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2281,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6579},{"shards":"0,2","objects":1884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12093","reported_seq":12275,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.979820+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:41.979820+0000","last_peered":"2024-09-11T01:19:41.979820+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:41.979820+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35033088,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8486,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4387},{"shards":"0,2","objects":2426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12014","reported_seq":12198,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.938570+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:41.938570+0000","last_peered":"2024-09-11T01:19:41.938570+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:41.938570+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8549,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11846","reported_seq":12007,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.939337+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:41.939337+0000","last_peered":"2024-09-11T01:19:41.939337+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:41.939337+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964998,"num_objects":8294,"num_object_clones":0,"num_object_copies":16588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8503,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":2383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554808054,"num_objects":135461,"num_object_clones":0,"num_object_copies":270922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41102,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":135461,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":149780,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6648,"num_bytes_recovered":26231267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1206575104,"data_stored":1206418827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160917,"ondisk_log_size":160917,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693204,"kb_used_data":404676,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93678636,"statfs":{"total":96636764160,"available":95926923264,"internally_reserved":0,"allocated":414388224,"data_stored":414116847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,1,0,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790005,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609732,"kb_used_data":303156,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93762108,"statfs":{"total":96636764160,"available":96012398592,"internally_reserved":0,"allocated":310431744,"data_stored":310112934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,9,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838280,"kb_used_data":473464,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93533560,"statfs":{"total":96636764160,"available":95778365440,"internally_reserved":0,"allocated":484827136,"data_stored":484555754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,4,3,2,1,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":483282944,"data_stored":483246814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309968896,"data_stored":309888951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":413323264,"data_stored":413283062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:46.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:46.722 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:46.724 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:46.744 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-11T01:19:46.526462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":551527177,"num_objects":134662,"num_object_clones":0,"num_object_copies":269324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41093,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":134662,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":150583,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6658,"num_bytes_recovered":26231286,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2144784,"kb_used_data":1176536,"kb_used_omap":12,"kb_used_meta":968179,"kb_avail":280970736,"statfs":{"total":289910292480,"available":287714033664,"internally_reserved":0,"allocated":1204772864,"data_stored":1203914908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":991415303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,3,6,3,5,1,0,0,4,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20250624,"num_objects":-4944,"num_object_clones":0,"num_object_copies":-9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-248,"num_objects_unfound":0,"num_objects_dirty":-4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4944,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1015808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008330"},"pg_stats":[{"pgid":"2.f","version":"41'11850","reported_seq":11989,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.866441+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:41.866441+0000","last_peered":"2024-09-11T01:19:41.866441+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:41.866441+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34357248,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11850,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":2392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12125","reported_seq":12201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.497199+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:45.497199+0000","last_peered":"2024-09-11T01:19:45.497199+0000","last_clean":"2024-09-11T01:19:45.497199+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:45.497199+0000","last_undegraded":"2024-09-11T01:19:45.497199+0000","last_fullsized":"2024-09-11T01:19:45.497199+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34395078,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10223,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11813","reported_seq":11943,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.526439+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:45.526439+0000","last_peered":"2024-09-11T01:19:45.526439+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:45.526439+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33718296,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9874,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2216},{"shards":"2","objects":4253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12026","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.415844+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:45.415844+0000","last_peered":"2024-09-11T01:19:45.415844+0000","last_clean":"2024-09-11T01:19:45.415844+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:45.415844+0000","last_undegraded":"2024-09-11T01:19:45.415844+0000","last_fullsized":"2024-09-11T01:19:45.415844+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12026,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11898","reported_seq":12015,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:41.784320+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:41.784320+0000","last_peered":"2024-09-11T01:19:41.784320+0000","last_clean":"2024-09-11T01:19:41.784320+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:41.784320+0000","last_undegraded":"2024-09-11T01:19:41.784320+0000","last_fullsized":"2024-09-11T01:19:41.784320+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34320867,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8696,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11850","reported_seq":12030,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.891543+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:41.891543+0000","last_peered":"2024-09-11T01:19:41.891543+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:41.891543+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8338,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4268},{"shards":"0,2","objects":2370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12161","reported_seq":12342,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.899769+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:41.899769+0000","last_peered":"2024-09-11T01:19:41.899769+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:41.899769+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35435462,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8914,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4396},{"shards":"0,2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11951","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.000011+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:42.000011+0000","last_peered":"2024-09-11T01:19:42.000011+0000","last_clean":"2024-09-11T01:19:42.000011+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:42.000011+0000","last_undegraded":"2024-09-11T01:19:42.000011+0000","last_fullsized":"2024-09-11T01:19:42.000011+0000","mapping_epoch":31,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8904,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11935","reported_seq":20273,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:44.475707+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:44.475707+0000","last_peered":"2024-09-11T01:19:44.475707+0000","last_clean":"2024-09-11T01:19:44.475707+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:44.475707+0000","last_undegraded":"2024-09-11T01:19:44.475707+0000","last_fullsized":"2024-09-11T01:19:44.475707+0000","mapping_epoch":40,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054627,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9802,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12004","reported_seq":12179,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.985498+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:41.985498+0000","last_peered":"2024-09-11T01:19:41.985498+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:41.985498+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35184640,"num_objects":8590,"num_object_clones":0,"num_object_copies":17180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8368,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4510},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11921","reported_seq":12057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.459602+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:45.459602+0000","last_peered":"2024-09-11T01:19:45.459602+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:45.459602+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661411,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9690,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2178},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12142","reported_seq":12273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.521744+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:45.521744+0000","last_peered":"2024-09-11T01:19:45.521744+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:45.521744+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":4264}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11889","reported_seq":16577,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:41.981872+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:41.981872+0000","last_peered":"2024-09-11T01:19:41.981872+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:41.981872+0000","last_undegraded":"2024-09-11T01:19:41.981872+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34664448,"num_objects":8463,"num_object_clones":0,"num_object_copies":16926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":8463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2281,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6579},{"shards":"0,2","objects":1884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12093","reported_seq":12275,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.979820+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:41.979820+0000","last_peered":"2024-09-11T01:19:41.979820+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:41.979820+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35033088,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8486,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4387},{"shards":"0,2","objects":2426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12014","reported_seq":12198,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.938570+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:41.938570+0000","last_peered":"2024-09-11T01:19:41.938570+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:41.938570+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8549,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11846","reported_seq":12007,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.939337+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:41.939337+0000","last_peered":"2024-09-11T01:19:41.939337+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:41.939337+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964998,"num_objects":8294,"num_object_clones":0,"num_object_copies":16588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8503,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":2383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":551527158,"num_objects":134660,"num_object_clones":0,"num_object_copies":269320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41093,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":134660,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":150581,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6657,"num_bytes_recovered":26231267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1201606656,"data_stored":1201450379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693144,"kb_used_data":400448,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93678696,"statfs":{"total":96636764160,"available":95926984704,"internally_reserved":0,"allocated":410058752,"data_stored":409787375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,3,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613360,"kb_used_data":302624,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93758480,"statfs":{"total":96636764160,"available":96008683520,"internally_reserved":0,"allocated":309886976,"data_stored":309571779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838280,"kb_used_data":473464,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93533560,"statfs":{"total":96636764160,"available":95778365440,"internally_reserved":0,"allocated":484827136,"data_stored":484555754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,4,3,2,1,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":483282944,"data_stored":483246814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309424128,"data_stored":309344183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408899584,"data_stored":408859382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:46.745 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:47.328 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:47.356 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-11T01:19:46.526462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":551527177,"num_objects":134662,"num_object_clones":0,"num_object_copies":269324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41093,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":134662,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":150583,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6658,"num_bytes_recovered":26231286,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2144784,"kb_used_data":1176536,"kb_used_omap":12,"kb_used_meta":968179,"kb_avail":280970736,"statfs":{"total":289910292480,"available":287714033664,"internally_reserved":0,"allocated":1204772864,"data_stored":1203914908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":991415303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,3,6,3,5,1,0,0,4,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20250624,"num_objects":-4944,"num_object_clones":0,"num_object_copies":-9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-248,"num_objects_unfound":0,"num_objects_dirty":-4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4944,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1015808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008330"},"pg_stats":[{"pgid":"2.f","version":"41'11850","reported_seq":11989,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.866441+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:41.866441+0000","last_peered":"2024-09-11T01:19:41.866441+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:41.866441+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34357248,"num_objects":8388,"num_object_clones":0,"num_object_copies":16776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11850,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4300},{"shards":"0,2","objects":2392}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12125","reported_seq":12201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.497199+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:45.497199+0000","last_peered":"2024-09-11T01:19:45.497199+0000","last_clean":"2024-09-11T01:19:45.497199+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:45.497199+0000","last_undegraded":"2024-09-11T01:19:45.497199+0000","last_fullsized":"2024-09-11T01:19:45.497199+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34395078,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10223,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11813","reported_seq":11943,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.526439+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:45.526439+0000","last_peered":"2024-09-11T01:19:45.526439+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:45.526439+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33718296,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9874,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2216},{"shards":"2","objects":4253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12026","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.415844+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:45.415844+0000","last_peered":"2024-09-11T01:19:45.415844+0000","last_clean":"2024-09-11T01:19:45.415844+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:45.415844+0000","last_undegraded":"2024-09-11T01:19:45.415844+0000","last_fullsized":"2024-09-11T01:19:45.415844+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12026,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'11898","reported_seq":12015,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:41.784320+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:41.784320+0000","last_peered":"2024-09-11T01:19:41.784320+0000","last_clean":"2024-09-11T01:19:41.784320+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:41.784320+0000","last_undegraded":"2024-09-11T01:19:41.784320+0000","last_fullsized":"2024-09-11T01:19:41.784320+0000","mapping_epoch":31,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34320867,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8696,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11850","reported_seq":12030,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.891543+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:41.891543+0000","last_peered":"2024-09-11T01:19:41.891543+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:41.891543+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230755,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8338,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4268},{"shards":"0,2","objects":2370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12161","reported_seq":12342,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.899769+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:41.899769+0000","last_peered":"2024-09-11T01:19:41.899769+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:41.899769+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35435462,"num_objects":8653,"num_object_clones":0,"num_object_copies":17306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8914,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4396},{"shards":"0,2","objects":2531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'11951","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.000011+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:42.000011+0000","last_peered":"2024-09-11T01:19:42.000011+0000","last_clean":"2024-09-11T01:19:42.000011+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:42.000011+0000","last_undegraded":"2024-09-11T01:19:42.000011+0000","last_fullsized":"2024-09-11T01:19:42.000011+0000","mapping_epoch":31,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8904,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11935","reported_seq":20273,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:44.475707+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:44.475707+0000","last_peered":"2024-09-11T01:19:44.475707+0000","last_clean":"2024-09-11T01:19:44.475707+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:44.475707+0000","last_undegraded":"2024-09-11T01:19:44.475707+0000","last_fullsized":"2024-09-11T01:19:44.475707+0000","mapping_epoch":40,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054627,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9802,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12004","reported_seq":12179,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.985498+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:41.985498+0000","last_peered":"2024-09-11T01:19:41.985498+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:41.985498+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35184640,"num_objects":8590,"num_object_clones":0,"num_object_copies":17180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8368,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4510},{"shards":"0,2","objects":2402}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11921","reported_seq":12057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.459602+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:45.459602+0000","last_peered":"2024-09-11T01:19:45.459602+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:45.459602+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661411,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9690,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2178},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12142","reported_seq":12273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.521744+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:45.521744+0000","last_peered":"2024-09-11T01:19:45.521744+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:45.521744+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":4264}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'11889","reported_seq":16577,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:41.981872+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:41.981872+0000","last_peered":"2024-09-11T01:19:41.981872+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:41.981872+0000","last_undegraded":"2024-09-11T01:19:41.981872+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34664448,"num_objects":8463,"num_object_clones":0,"num_object_copies":16926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":8463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8579,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2281,"num_bytes_recovered":9338880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6579},{"shards":"0,2","objects":1884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12093","reported_seq":12275,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.979820+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:41.979820+0000","last_peered":"2024-09-11T01:19:41.979820+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:41.979820+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35033088,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8486,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4387},{"shards":"0,2","objects":2426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12014","reported_seq":12198,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.938570+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:41.938570+0000","last_peered":"2024-09-11T01:19:41.938570+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:41.938570+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8549,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4466},{"shards":"0,2","objects":2436}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11846","reported_seq":12007,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:41.939337+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:41.939337+0000","last_peered":"2024-09-11T01:19:41.939337+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:41.939337+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33964998,"num_objects":8294,"num_object_clones":0,"num_object_copies":16588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8503,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":2383}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":551527158,"num_objects":134660,"num_object_clones":0,"num_object_copies":269320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41093,"num_objects_misplaced":1884,"num_objects_unfound":0,"num_objects_dirty":134660,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":150581,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6657,"num_bytes_recovered":26231267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1201606656,"data_stored":1201450379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160718,"ondisk_log_size":160718,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693144,"kb_used_data":400448,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93678696,"statfs":{"total":96636764160,"available":95926984704,"internally_reserved":0,"allocated":410058752,"data_stored":409787375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,3,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613360,"kb_used_data":302624,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93758480,"statfs":{"total":96636764160,"available":96008683520,"internally_reserved":0,"allocated":309886976,"data_stored":309571779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838280,"kb_used_data":473464,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93533560,"statfs":{"total":96636764160,"available":95778365440,"internally_reserved":0,"allocated":484827136,"data_stored":484555754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,4,3,2,1,0,0,4],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":483282944,"data_stored":483246814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309424128,"data_stored":309344183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408899584,"data_stored":408859382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'11850", 'reported_seq': 11989, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.866441+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:41.866441+0000', 'last_peered': '2024-09-11T01:19:41.866441+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:41.866441+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34357248, 'num_objects': 8388, 'num_object_clones': 0, 'num_object_copies': 16776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11850, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4300}, {'shards': '0,2', 'objects': 2392}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11813", 'reported_seq': 11943, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:45.526439+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:45.526439+0000', 'last_peered': '2024-09-11T01:19:45.526439+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:45.526439+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33718296, 'num_objects': 8233, 'num_object_clones': 0, 'num_object_copies': 16466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3981, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8233, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9874, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2216}, {'shards': '2', 'objects': 4253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'11850", 'reported_seq': 12030, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.891543+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:41.891543+0000', 'last_peered': '2024-09-11T01:19:41.891543+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:41.891543+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34230755, 'num_objects': 8358, 'num_object_clones': 0, 'num_object_copies': 16716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8358, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8338, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4268}, {'shards': '0,2', 'objects': 2370}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'12161", 'reported_seq': 12342, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.899769+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:41.899769+0000', 'last_peered': '2024-09-11T01:19:41.899769+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:41.899769+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35435462, 'num_objects': 8653, 'num_object_clones': 0, 'num_object_copies': 17306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8653, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 8914, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4396}, {'shards': '0,2', 'objects': 2531}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:47.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12004", 'reported_seq': 12179, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.985498+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:41.985498+0000', 'last_peered': '2024-09-11T01:19:41.985498+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:41.985498+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2000", 'ondisk_log_start': "32'2000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35184640, 'num_objects': 8590, 'num_object_clones': 0, 'num_object_copies': 17180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4082, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8368, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4510}, {'shards': '0,2', 'objects': 2402}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'11921", 'reported_seq': 12057, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:45.459602+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:45.459602+0000', 'last_peered': '2024-09-11T01:19:45.459602+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:45.459602+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'1900", 'ondisk_log_start': "32'1900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33661411, 'num_objects': 8219, 'num_object_clones': 0, 'num_object_copies': 16438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4005, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8219, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9690, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2178}, {'shards': '2', 'objects': 4215}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12142", 'reported_seq': 12273, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:45.521744+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:45.521744+0000', 'last_peered': '2024-09-11T01:19:45.521744+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:45.521744+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34488320, 'num_objects': 8420, 'num_object_clones': 0, 'num_object_copies': 16840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4158, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9779, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2319}, {'shards': '2', 'objects': 4264}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'11889", 'reported_seq': 16577, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:41.981872+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:41.981872+0000', 'last_peered': '2024-09-11T01:19:41.981872+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:41.981872+0000', 'last_undegraded': '2024-09-11T01:19:41.981872+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34664448, 'num_objects': 8463, 'num_object_clones': 0, 'num_object_copies': 16926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1884, 'num_objects_unfound': 0, 'num_objects_dirty': 8463, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8579, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2281, 'num_bytes_recovered': 9338880, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6579}, {'shards': '0,2', 'objects': 1884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'12093", 'reported_seq': 12275, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.979820+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:41.979820+0000', 'last_peered': '2024-09-11T01:19:41.979820+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:41.979820+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'2000", 'ondisk_log_start': "32'2000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35033088, 'num_objects': 8553, 'num_object_clones': 0, 'num_object_copies': 17106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4169, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8553, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8486, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4387}, {'shards': '0,2', 'objects': 2426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:47.359 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'12014", 'reported_seq': 12198, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.938570+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:41.938570+0000', 'last_peered': '2024-09-11T01:19:41.938570+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:41.938570+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'2000", 'ondisk_log_start': "32'2000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35168256, 'num_objects': 8586, 'num_object_clones': 0, 'num_object_copies': 17172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4122, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8549, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4466}, {'shards': '0,2', 'objects': 2436}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.360 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:47.360 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'11846", 'reported_seq': 12007, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:41.939337+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:41.939337+0000', 'last_peered': '2024-09-11T01:19:41.939337+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:41.939337+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'1800", 'ondisk_log_start': "32'1800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33964998, 'num_objects': 8294, 'num_object_clones': 0, 'num_object_copies': 16588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4140, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8294, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8503, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 16, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4156}, {'shards': '0,2', 'objects': 2383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:47.360 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:48.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:48.100 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41093/269324 objects degraded (15.258%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":388,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":134662,"data_bytes":551527177,"bytes_used":2196258816,"bytes_avail":287714033664,"bytes_total":289910292480,"degraded_objects":41093,"degraded_total":269324,"degraded_ratio":0.15257830716906032,"misplaced_objects":1884,"misplaced_total":269324,"misplaced_ratio":0.0069952919160564979,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":84591,"recovering_keys_per_sec":0,"num_objects_recovered":279,"num_bytes_recovered":1015808,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":411},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:48.122 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:48.122 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:48.714 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:48.715 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:48.745 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T01:19:48.530229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545903369,"num_objects":133289,"num_object_clones":0,"num_object_copies":266578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41079,"num_objects_misplaced":1764,"num_objects_unfound":0,"num_objects_dirty":133289,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":151956,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6792,"num_bytes_recovered":26722806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160694,"ondisk_log_size":160694,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2129872,"kb_used_data":1169944,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":280985648,"statfs":{"total":289910292480,"available":287729303552,"internally_reserved":0,"allocated":1198022656,"data_stored":1197164700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":982895623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,7,2,3,0,0,0,8,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23089152,"num_objects":-5637,"num_object_clones":0,"num_object_copies":-11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-368,"num_objects_unfound":0,"num_objects_dirty":-5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5637,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1507328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007725"},"pg_stats":[{"pgid":"2.f","version":"41'11974","reported_seq":12120,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.998417+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:46.998417+0000","last_peered":"2024-09-11T01:19:46.998417+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:46.998417+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33849344,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11974,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":2268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12125","reported_seq":12201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.497199+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:45.497199+0000","last_peered":"2024-09-11T01:19:45.497199+0000","last_clean":"2024-09-11T01:19:45.497199+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:45.497199+0000","last_undegraded":"2024-09-11T01:19:45.497199+0000","last_fullsized":"2024-09-11T01:19:45.497199+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34395078,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10223,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11813","reported_seq":11943,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.526439+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:45.526439+0000","last_peered":"2024-09-11T01:19:45.526439+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:45.526439+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33718296,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9874,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2216},{"shards":"2","objects":4253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12026","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.415844+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:45.415844+0000","last_peered":"2024-09-11T01:19:45.415844+0000","last_clean":"2024-09-11T01:19:45.415844+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:45.415844+0000","last_undegraded":"2024-09-11T01:19:45.415844+0000","last_fullsized":"2024-09-11T01:19:45.415844+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12026,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12032","reported_seq":12149,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.834574+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:46.834574+0000","last_peered":"2024-09-11T01:19:46.834574+0000","last_clean":"2024-09-11T01:19:46.834574+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:46.834574+0000","last_undegraded":"2024-09-11T01:19:46.834574+0000","last_fullsized":"2024-09-11T01:19:46.834574+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33772003,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8830,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11965","reported_seq":12151,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.940554+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:46.940554+0000","last_peered":"2024-09-11T01:19:46.940554+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:46.940554+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33759715,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8453,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":2255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12330","reported_seq":12517,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.903990+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:46.903990+0000","last_peered":"2024-09-11T01:19:46.903990+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:46.903990+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34743238,"num_objects":8484,"num_object_clones":0,"num_object_copies":16968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8484,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9083,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12084","reported_seq":12199,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.982693+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:46.982693+0000","last_peered":"2024-09-11T01:19:46.982693+0000","last_clean":"2024-09-11T01:19:46.982693+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:46.982693+0000","last_undegraded":"2024-09-11T01:19:46.982693+0000","last_fullsized":"2024-09-11T01:19:46.982693+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'11935","reported_seq":20273,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:44.475707+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:44.475707+0000","last_peered":"2024-09-11T01:19:44.475707+0000","last_clean":"2024-09-11T01:19:44.475707+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:44.475707+0000","last_undegraded":"2024-09-11T01:19:44.475707+0000","last_fullsized":"2024-09-11T01:19:44.475707+0000","mapping_epoch":40,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054627,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9802,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12142","reported_seq":12319,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.985832+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:46.985832+0000","last_peered":"2024-09-11T01:19:46.985832+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:46.985832+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8506,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4373},{"shards":"0,2","objects":2264}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11921","reported_seq":12057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.459602+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:45.459602+0000","last_peered":"2024-09-11T01:19:45.459602+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:45.459602+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661411,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9690,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2178},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12142","reported_seq":12273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.521744+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:45.521744+0000","last_peered":"2024-09-11T01:19:45.521744+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:45.521744+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":4264}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12028","reported_seq":16956,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:46.999496+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:46.999496+0000","last_peered":"2024-09-11T01:19:46.999496+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:46.999496+0000","last_undegraded":"2024-09-11T01:19:46.999496+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1764,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2401,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6560},{"shards":"0,2","objects":1764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12231","reported_seq":12415,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.925661+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:46.925661+0000","last_peered":"2024-09-11T01:19:46.925661+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:46.925661+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8624,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4250},{"shards":"0,2","objects":2288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12165","reported_seq":12351,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.953042+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:46.953042+0000","last_peered":"2024-09-11T01:19:46.953042+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:46.953042+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8700,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4316},{"shards":"0,2","objects":2285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11978","reported_seq":12143,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.927912+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:46.927912+0000","last_peered":"2024-09-11T01:19:46.927912+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:46.927912+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33424326,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8635,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4025},{"shards":"0,2","objects":2251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545903350,"num_objects":133287,"num_object_clones":0,"num_object_copies":266574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41079,"num_objects_misplaced":1764,"num_objects_unfound":0,"num_objects_dirty":133287,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":151954,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":26722787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1194860544,"data_stored":1194704267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160691,"ondisk_log_size":160691,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693144,"kb_used_data":400448,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93678696,"statfs":{"total":96636764160,"available":95926984704,"internally_reserved":0,"allocated":410058752,"data_stored":409787375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,3,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790006,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613360,"kb_used_data":302624,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93758480,"statfs":{"total":96636764160,"available":96008683520,"internally_reserved":0,"allocated":309886976,"data_stored":309571779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":823368,"kb_used_data":466872,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93548472,"statfs":{"total":96636764160,"available":95793635328,"internally_reserved":0,"allocated":478076928,"data_stored":477805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,2,0,0,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":476536832,"data_stored":476500702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309424128,"data_stored":309344183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408899584,"data_stored":408859382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:51.748 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:52.317 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:52.319 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:52.341 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-11T01:19:50.531070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545420041,"num_objects":133171,"num_object_clones":0,"num_object_copies":266342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41079,"num_objects_misplaced":1764,"num_objects_unfound":0,"num_objects_dirty":133171,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":152074,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6792,"num_bytes_recovered":26722806,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160712,"ondisk_log_size":160712,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120928,"kb_used_data":1169320,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":280994592,"statfs":{"total":289910292480,"available":287738462208,"internally_reserved":0,"allocated":1197383680,"data_stored":1196525724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":974375943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,9,3,4,0,0,0,8,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18182144,"num_objects":-4439,"num_object_clones":0,"num_object_copies":-8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":-4439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4439,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007888"},"pg_stats":[{"pgid":"2.f","version":"41'11974","reported_seq":12120,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.998417+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:46.998417+0000","last_peered":"2024-09-11T01:19:46.998417+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:46.998417+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33849344,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11974,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4180},{"shards":"0,2","objects":2268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12125","reported_seq":12201,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.497199+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:45.497199+0000","last_peered":"2024-09-11T01:19:45.497199+0000","last_clean":"2024-09-11T01:19:45.497199+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:45.497199+0000","last_undegraded":"2024-09-11T01:19:45.497199+0000","last_fullsized":"2024-09-11T01:19:45.497199+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34395078,"num_objects":8399,"num_object_clones":0,"num_object_copies":16798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8399,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10223,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11813","reported_seq":11943,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.526439+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:45.526439+0000","last_peered":"2024-09-11T01:19:45.526439+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:45.526439+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1800","ondisk_log_start":"32'1800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33718296,"num_objects":8233,"num_object_clones":0,"num_object_copies":16466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9874,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2216},{"shards":"2","objects":4253}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12026","reported_seq":12066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:45.415844+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:45.415844+0000","last_peered":"2024-09-11T01:19:45.415844+0000","last_clean":"2024-09-11T01:19:45.415844+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:45.415844+0000","last_undegraded":"2024-09-11T01:19:45.415844+0000","last_fullsized":"2024-09-11T01:19:45.415844+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34144256,"num_objects":8336,"num_object_clones":0,"num_object_copies":16672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12026,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12032","reported_seq":12149,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.834574+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:46.834574+0000","last_peered":"2024-09-11T01:19:46.834574+0000","last_clean":"2024-09-11T01:19:46.834574+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:46.834574+0000","last_undegraded":"2024-09-11T01:19:46.834574+0000","last_fullsized":"2024-09-11T01:19:46.834574+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33772003,"num_objects":8246,"num_object_clones":0,"num_object_copies":16492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8830,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'11965","reported_seq":12151,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.940554+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:46.940554+0000","last_peered":"2024-09-11T01:19:46.940554+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:46.940554+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33759715,"num_objects":8243,"num_object_clones":0,"num_object_copies":16486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8453,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4156},{"shards":"0,2","objects":2255}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12330","reported_seq":12517,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.903990+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:46.903990+0000","last_peered":"2024-09-11T01:19:46.903990+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:46.903990+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34743238,"num_objects":8484,"num_object_clones":0,"num_object_copies":16968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4254,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8484,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9083,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":2362}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12084","reported_seq":12199,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.982693+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:46.982693+0000","last_peered":"2024-09-11T01:19:46.982693+0000","last_clean":"2024-09-11T01:19:46.982693+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:46.982693+0000","last_undegraded":"2024-09-11T01:19:46.982693+0000","last_fullsized":"2024-09-11T01:19:46.982693+0000","mapping_epoch":31,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34160640,"num_objects":8340,"num_object_clones":0,"num_object_copies":16680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9037,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12053","reported_seq":20392,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.455319+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:49.455319+0000","last_peered":"2024-09-11T01:19:49.455319+0000","last_clean":"2024-09-11T01:19:49.455319+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:49.455319+0000","last_undegraded":"2024-09-11T01:19:49.455319+0000","last_fullsized":"2024-09-11T01:19:49.455319+0000","mapping_epoch":40,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33571299,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9920,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12142","reported_seq":12319,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.985832+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:46.985832+0000","last_peered":"2024-09-11T01:19:46.985832+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:46.985832+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34619392,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8506,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4373},{"shards":"0,2","objects":2264}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'11921","reported_seq":12057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.459602+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:45.459602+0000","last_peered":"2024-09-11T01:19:45.459602+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:45.459602+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661411,"num_objects":8219,"num_object_clones":0,"num_object_copies":16438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8219,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9690,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2178},{"shards":"2","objects":4215}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12142","reported_seq":12273,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:45.521744+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:45.521744+0000","last_peered":"2024-09-11T01:19:45.521744+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:45.521744+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":4264}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12028","reported_seq":16956,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:46.999496+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:46.999496+0000","last_peered":"2024-09-11T01:19:46.999496+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:46.999496+0000","last_undegraded":"2024-09-11T01:19:46.999496+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34095104,"num_objects":8324,"num_object_clones":0,"num_object_copies":16648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1764,"num_objects_unfound":0,"num_objects_dirty":8324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2401,"num_bytes_recovered":9830400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6560},{"shards":"0,2","objects":1764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12231","reported_seq":12415,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.925661+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:46.925661+0000","last_peered":"2024-09-11T01:19:46.925661+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:46.925661+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8624,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4250},{"shards":"0,2","objects":2288}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12165","reported_seq":12351,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.953042+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:46.953042+0000","last_peered":"2024-09-11T01:19:46.953042+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:46.953042+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8700,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4316},{"shards":"0,2","objects":2285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'11978","reported_seq":12143,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:46.927912+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:46.927912+0000","last_peered":"2024-09-11T01:19:46.927912+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:46.927912+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33424326,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8635,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4025},{"shards":"0,2","objects":2251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545420022,"num_objects":133169,"num_object_clones":0,"num_object_copies":266338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41079,"num_objects_misplaced":1764,"num_objects_unfound":0,"num_objects_dirty":133169,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":152072,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6791,"num_bytes_recovered":26722787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1194270720,"data_stored":1194114443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160709,"ondisk_log_size":160709,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693144,"kb_used_data":400448,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93678696,"statfs":{"total":96636764160,"available":95926984704,"internally_reserved":0,"allocated":410058752,"data_stored":409787375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,0,3,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790007,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604416,"kb_used_data":302000,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93767424,"statfs":{"total":96636764160,"available":96017842176,"internally_reserved":0,"allocated":309248000,"data_stored":308932803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":823368,"kb_used_data":466872,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93548472,"statfs":{"total":96636764160,"available":95793635328,"internally_reserved":0,"allocated":478076928,"data_stored":477805546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,2,0,0,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":476536832,"data_stored":476500702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308834304,"data_stored":308754359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":408899584,"data_stored":408859382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:52.343 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:52.893 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:52.893 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:52.918 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-11T01:19:52.531565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537113353,"num_objects":131143,"num_object_clones":0,"num_object_copies":262286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41054,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":131143,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154102,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6936,"num_bytes_recovered":27210230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2101672,"kb_used_data":1158384,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":281013848,"statfs":{"total":289910292480,"available":287758180352,"internally_reserved":0,"allocated":1186185216,"data_stored":1185327260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":965856263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,7,7,5,3,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25915392,"num_objects":-6327,"num_object_clones":0,"num_object_copies":-12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-364,"num_objects_unfound":0,"num_objects_dirty":-6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6327,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":1490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007467"},"pg_stats":[{"pgid":"2.f","version":"41'12102","reported_seq":12251,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008800+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:52.008800+0000","last_peered":"2024-09-11T01:19:52.008800+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:52.008800+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12102,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4053},{"shards":"0,2","objects":2140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12255","reported_seq":12331,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:50.514997+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:50.514997+0000","last_peered":"2024-09-11T01:19:50.514997+0000","last_clean":"2024-09-11T01:19:50.514997+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:50.514997+0000","last_undegraded":"2024-09-11T01:19:50.514997+0000","last_fullsized":"2024-09-11T01:19:50.514997+0000","mapping_epoch":31,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33862598,"num_objects":8269,"num_object_clones":0,"num_object_copies":16538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8269,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10353,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11954","reported_seq":12084,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:50.536367+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:50.536367+0000","last_peered":"2024-09-11T01:19:50.536367+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:50.536367+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140760,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10015,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2075},{"shards":"2","objects":4112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12168","reported_seq":12209,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:50.538471+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:50.538471+0000","last_peered":"2024-09-11T01:19:50.538471+0000","last_clean":"2024-09-11T01:19:50.538471+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:50.538471+0000","last_undegraded":"2024-09-11T01:19:50.538471+0000","last_fullsized":"2024-09-11T01:19:50.538471+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562624,"num_objects":8194,"num_object_clones":0,"num_object_copies":16388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12168,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12176","reported_seq":12293,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.901770+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:51.901770+0000","last_peered":"2024-09-11T01:19:51.901770+0000","last_clean":"2024-09-11T01:19:51.901770+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:51.901770+0000","last_undegraded":"2024-09-11T01:19:51.901770+0000","last_fullsized":"2024-09-11T01:19:51.901770+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33182179,"num_objects":8102,"num_object_clones":0,"num_object_copies":16204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8974,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12116","reported_seq":12306,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008942+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:52.008942+0000","last_peered":"2024-09-11T01:19:52.008942+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:52.008942+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8604,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4007},{"shards":"0,2","objects":2104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12459","reported_seq":12660,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.989862+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:51.989862+0000","last_peered":"2024-09-11T01:19:51.989862+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:51.989862+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34214854,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9212,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12227","reported_seq":12342,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.942671+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:51.942671+0000","last_peered":"2024-09-11T01:19:51.942671+0000","last_clean":"2024-09-11T01:19:51.942671+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:51.942671+0000","last_undegraded":"2024-09-11T01:19:51.942671+0000","last_fullsized":"2024-09-11T01:19:51.942671+0000","mapping_epoch":31,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33574912,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9180,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12053","reported_seq":20392,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.455319+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:49.455319+0000","last_peered":"2024-09-11T01:19:49.455319+0000","last_clean":"2024-09-11T01:19:49.455319+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:49.455319+0000","last_undegraded":"2024-09-11T01:19:49.455319+0000","last_fullsized":"2024-09-11T01:19:49.455319+0000","mapping_epoch":40,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33571299,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9920,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12287","reported_seq":12468,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.007362+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:52.007362+0000","last_peered":"2024-09-11T01:19:52.007362+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:52.007362+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8651,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":2119}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12030","reported_seq":12166,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:50.461114+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:50.461114+0000","last_peered":"2024-09-11T01:19:50.461114+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:50.461114+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33214947,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9799,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2069},{"shards":"2","objects":4106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12267","reported_seq":12398,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:50.356502+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:50.356502+0000","last_peered":"2024-09-11T01:19:50.356502+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:50.356502+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976320,"num_objects":8295,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9904,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2194},{"shards":"2","objects":4139}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12167","reported_seq":17334,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:52.012628+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:52.012628+0000","last_peered":"2024-09-11T01:19:52.012628+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:52.012628+0000","last_undegraded":"2024-09-11T01:19:52.012628+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8857,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2520,"num_bytes_recovered":10317824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6540},{"shards":"0,2","objects":1645}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12346","reported_seq":12542,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.936973+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:51.936973+0000","last_peered":"2024-09-11T01:19:51.936973+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:51.936973+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8739,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4141},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12326","reported_seq":12527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.897971+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:51.897971+0000","last_peered":"2024-09-11T01:19:51.897971+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:51.897971+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12104","reported_seq":12269,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.922792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:51.922792+0000","last_peered":"2024-09-11T01:19:51.922792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:51.922792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32908230,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8761,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537113334,"num_objects":131141,"num_object_clones":0,"num_object_copies":262282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41054,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":131141,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154100,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6935,"num_bytes_recovered":27210211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1183182848,"data_stored":1183026571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676536,"kb_used_data":396328,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93695304,"statfs":{"total":96636764160,"available":95943991296,"internally_reserved":0,"allocated":405839872,"data_stored":405568495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790007,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604416,"kb_used_data":302000,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93767424,"statfs":{"total":96636764160,"available":96017842176,"internally_reserved":0,"allocated":309248000,"data_stored":308932803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820720,"kb_used_data":460056,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93551120,"statfs":{"total":96636764160,"available":95796346880,"internally_reserved":0,"allocated":471097344,"data_stored":470825962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469729280,"data_stored":469693150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308834304,"data_stored":308754359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404619264,"data_stored":404579062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:52.919 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12102", 'reported_seq': 12251, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:52.008800+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:52.008800+0000', 'last_peered': '2024-09-11T01:19:52.008800+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:52.008800+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33325056, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4083, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12102, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4053}, {'shards': '0,2', 'objects': 2140}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'11954", 'reported_seq': 12084, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:50.536367+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:50.536367+0000', 'last_peered': '2024-09-11T01:19:50.536367+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:50.536367+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'1900", 'ondisk_log_start': "32'1900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33140760, 'num_objects': 8092, 'num_object_clones': 0, 'num_object_copies': 16184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3981, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8092, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10015, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2075}, {'shards': '2', 'objects': 4112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12116", 'reported_seq': 12306, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:52.008942+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:52.008942+0000', 'last_peered': '2024-09-11T01:19:52.008942+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:52.008942+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33141219, 'num_objects': 8092, 'num_object_clones': 0, 'num_object_copies': 16184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4086, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8092, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8604, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4007}, {'shards': '0,2', 'objects': 2104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'12459", 'reported_seq': 12660, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.989862+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:51.989862+0000', 'last_peered': '2024-09-11T01:19:51.989862+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:51.989862+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34214854, 'num_objects': 8355, 'num_object_clones': 0, 'num_object_copies': 16710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4247, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8355, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9212, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4108}, {'shards': '0,2', 'objects': 2233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12287", 'reported_seq': 12468, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:52.007362+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:52.007362+0000', 'last_peered': '2024-09-11T01:19:52.007362+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:52.007362+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2200", 'ondisk_log_start': "32'2200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34025472, 'num_objects': 8307, 'num_object_clones': 0, 'num_object_copies': 16614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8651, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4230}, {'shards': '0,2', 'objects': 2119}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.920 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12030", 'reported_seq': 12166, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:50.461114+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:50.461114+0000', 'last_peered': '2024-09-11T01:19:50.461114+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:50.461114+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2000", 'ondisk_log_start': "32'2000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33214947, 'num_objects': 8110, 'num_object_clones': 0, 'num_object_copies': 16220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4005, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8110, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9799, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2069}, {'shards': '2', 'objects': 4106}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12267", 'reported_seq': 12398, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:50.356502+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:50.356502+0000', 'last_peered': '2024-09-11T01:19:50.356502+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:50.356502+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2200", 'ondisk_log_start': "32'2200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33976320, 'num_objects': 8295, 'num_object_clones': 0, 'num_object_copies': 16590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4158, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9904, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 21, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2194}, {'shards': '2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'12167", 'reported_seq': 17334, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:52.012628+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:52.012628+0000', 'last_peered': '2024-09-11T01:19:52.012628+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:52.012628+0000', 'last_undegraded': '2024-09-11T01:19:52.012628+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33525760, 'num_objects': 8185, 'num_object_clones': 0, 'num_object_copies': 16370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1645, 'num_objects_unfound': 0, 'num_objects_dirty': 8185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8857, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2520, 'num_bytes_recovered': 10317824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6540}, {'shards': '0,2', 'objects': 1645}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'12346", 'reported_seq': 12542, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.936973+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:51.936973+0000', 'last_peered': '2024-09-11T01:19:51.936973+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:51.936973+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33996800, 'num_objects': 8300, 'num_object_clones': 0, 'num_object_copies': 16600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4162, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8739, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4141}, {'shards': '0,2', 'objects': 2173}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'12326", 'reported_seq': 12527, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.897971+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:51.897971+0000', 'last_peered': '2024-09-11T01:19:51.897971+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:51.897971+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33890304, 'num_objects': 8274, 'num_object_clones': 0, 'num_object_copies': 16548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8861, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4162}, {'shards': '0,2', 'objects': 2124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:52.921 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12104", 'reported_seq': 12269, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.922792+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:51.922792+0000', 'last_peered': '2024-09-11T01:19:51.922792+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:51.922792+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32908230, 'num_objects': 8036, 'num_object_clones': 0, 'num_object_copies': 16072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8036, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8761, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3899}, {'shards': '0,2', 'objects': 2125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.922 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:53.614 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:53.614 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41054/262286 objects degraded (15.652%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":394,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131143,"data_bytes":537113353,"bytes_used":2152112128,"bytes_avail":287758180352,"bytes_total":289910292480,"degraded_objects":41054,"degraded_total":262286,"degraded_ratio":0.1565237946363893,"misplaced_objects":1645,"misplaced_total":262286,"misplaced_ratio":0.0062717796603707406,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":124168,"recovering_keys_per_sec":0,"num_objects_recovered":422,"num_bytes_recovered":1490944,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":526},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:53.646 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:53.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:54.232 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:54.233 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:54.257 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-11T01:19:52.531565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537113353,"num_objects":131143,"num_object_clones":0,"num_object_copies":262286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41054,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":131143,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154102,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6936,"num_bytes_recovered":27210230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160740,"ondisk_log_size":160740,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2101672,"kb_used_data":1158384,"kb_used_omap":12,"kb_used_meta":943219,"kb_avail":281013848,"statfs":{"total":289910292480,"available":287758180352,"internally_reserved":0,"allocated":1186185216,"data_stored":1185327260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":965856263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,7,7,5,3,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25915392,"num_objects":-6327,"num_object_clones":0,"num_object_copies":-12654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-364,"num_objects_unfound":0,"num_objects_dirty":-6327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6327,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":1490944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007467"},"pg_stats":[{"pgid":"2.f","version":"41'12102","reported_seq":12251,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008800+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:52.008800+0000","last_peered":"2024-09-11T01:19:52.008800+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:52.008800+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12102,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4053},{"shards":"0,2","objects":2140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12255","reported_seq":12331,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:50.514997+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:50.514997+0000","last_peered":"2024-09-11T01:19:50.514997+0000","last_clean":"2024-09-11T01:19:50.514997+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:50.514997+0000","last_undegraded":"2024-09-11T01:19:50.514997+0000","last_fullsized":"2024-09-11T01:19:50.514997+0000","mapping_epoch":31,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33862598,"num_objects":8269,"num_object_clones":0,"num_object_copies":16538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8269,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10353,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'11954","reported_seq":12084,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:50.536367+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:50.536367+0000","last_peered":"2024-09-11T01:19:50.536367+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:50.536367+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'1900","ondisk_log_start":"32'1900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33140760,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3981,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10015,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2075},{"shards":"2","objects":4112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12168","reported_seq":12209,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:50.538471+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:50.538471+0000","last_peered":"2024-09-11T01:19:50.538471+0000","last_clean":"2024-09-11T01:19:50.538471+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:50.538471+0000","last_undegraded":"2024-09-11T01:19:50.538471+0000","last_fullsized":"2024-09-11T01:19:50.538471+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562624,"num_objects":8194,"num_object_clones":0,"num_object_copies":16388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12168,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12176","reported_seq":12293,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.901770+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:51.901770+0000","last_peered":"2024-09-11T01:19:51.901770+0000","last_clean":"2024-09-11T01:19:51.901770+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:51.901770+0000","last_undegraded":"2024-09-11T01:19:51.901770+0000","last_fullsized":"2024-09-11T01:19:51.901770+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33182179,"num_objects":8102,"num_object_clones":0,"num_object_copies":16204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8974,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12116","reported_seq":12306,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008942+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:52.008942+0000","last_peered":"2024-09-11T01:19:52.008942+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:52.008942+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8604,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4007},{"shards":"0,2","objects":2104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12459","reported_seq":12660,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.989862+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:51.989862+0000","last_peered":"2024-09-11T01:19:51.989862+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:51.989862+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34214854,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9212,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12227","reported_seq":12342,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.942671+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:51.942671+0000","last_peered":"2024-09-11T01:19:51.942671+0000","last_clean":"2024-09-11T01:19:51.942671+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:51.942671+0000","last_undegraded":"2024-09-11T01:19:51.942671+0000","last_fullsized":"2024-09-11T01:19:51.942671+0000","mapping_epoch":31,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33574912,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9180,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12053","reported_seq":20392,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.455319+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:49.455319+0000","last_peered":"2024-09-11T01:19:49.455319+0000","last_clean":"2024-09-11T01:19:49.455319+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:49.455319+0000","last_undegraded":"2024-09-11T01:19:49.455319+0000","last_fullsized":"2024-09-11T01:19:49.455319+0000","mapping_epoch":40,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33571299,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9920,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12287","reported_seq":12468,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.007362+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:52.007362+0000","last_peered":"2024-09-11T01:19:52.007362+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:52.007362+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8651,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":2119}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12030","reported_seq":12166,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:50.461114+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:50.461114+0000","last_peered":"2024-09-11T01:19:50.461114+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:50.461114+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33214947,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9799,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2069},{"shards":"2","objects":4106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12267","reported_seq":12398,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:50.356502+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:50.356502+0000","last_peered":"2024-09-11T01:19:50.356502+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:50.356502+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976320,"num_objects":8295,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4158,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9904,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2194},{"shards":"2","objects":4139}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12167","reported_seq":17334,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:52.012628+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:52.012628+0000","last_peered":"2024-09-11T01:19:52.012628+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:52.012628+0000","last_undegraded":"2024-09-11T01:19:52.012628+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8857,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2520,"num_bytes_recovered":10317824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6540},{"shards":"0,2","objects":1645}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12346","reported_seq":12542,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.936973+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:51.936973+0000","last_peered":"2024-09-11T01:19:51.936973+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:51.936973+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8739,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4141},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12326","reported_seq":12527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.897971+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:51.897971+0000","last_peered":"2024-09-11T01:19:51.897971+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:51.897971+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12104","reported_seq":12269,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.922792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:51.922792+0000","last_peered":"2024-09-11T01:19:51.922792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:51.922792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32908230,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8761,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537113334,"num_objects":131141,"num_object_clones":0,"num_object_copies":262282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41054,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":131141,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154100,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6935,"num_bytes_recovered":27210211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1183182848,"data_stored":1183026571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676536,"kb_used_data":396328,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93695304,"statfs":{"total":96636764160,"available":95943991296,"internally_reserved":0,"allocated":405839872,"data_stored":405568495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790007,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604416,"kb_used_data":302000,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93767424,"statfs":{"total":96636764160,"available":96017842176,"internally_reserved":0,"allocated":309248000,"data_stored":308932803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820720,"kb_used_data":460056,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93551120,"statfs":{"total":96636764160,"available":95796346880,"internally_reserved":0,"allocated":471097344,"data_stored":470825962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469729280,"data_stored":469693150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308834304,"data_stored":308754359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404619264,"data_stored":404579062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:57.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:57.846 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:57.847 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:57.875 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-11T01:19:56.532941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533795593,"num_objects":130333,"num_object_clones":0,"num_object_copies":260666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41052,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":130333,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154912,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6938,"num_bytes_recovered":27210230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105112,"kb_used_data":1153504,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":281010408,"statfs":{"total":289910292480,"available":287754657792,"internally_reserved":0,"allocated":1181188096,"data_stored":1180330140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":974375943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,6,4,6,0,0,6,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20537344,"num_objects":-5014,"num_object_clones":0,"num_object_copies":-10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-239,"num_objects_unfound":0,"num_objects_dirty":-5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007362"},"pg_stats":[{"pgid":"2.f","version":"41'12102","reported_seq":12251,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008800+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:52.008800+0000","last_peered":"2024-09-11T01:19:52.008800+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:52.008800+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12102,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4053},{"shards":"0,2","objects":2140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12383","reported_seq":12459,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:55.524304+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:55.524304+0000","last_peered":"2024-09-11T01:19:55.524304+0000","last_clean":"2024-09-11T01:19:55.524304+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:55.524304+0000","last_undegraded":"2024-09-11T01:19:55.524304+0000","last_fullsized":"2024-09-11T01:19:55.524304+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33338310,"num_objects":8141,"num_object_clones":0,"num_object_copies":16282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12099","reported_seq":12231,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.496662+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:55.496662+0000","last_peered":"2024-09-11T01:19:55.496662+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:55.496662+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546840,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10160,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":3968}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12312","reported_seq":12353,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:55.502626+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:55.502626+0000","last_peered":"2024-09-11T01:19:55.502626+0000","last_clean":"2024-09-11T01:19:55.502626+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:55.502626+0000","last_undegraded":"2024-09-11T01:19:55.502626+0000","last_fullsized":"2024-09-11T01:19:55.502626+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32972800,"num_objects":8050,"num_object_clones":0,"num_object_copies":16100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12312,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12176","reported_seq":12293,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.901770+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:51.901770+0000","last_peered":"2024-09-11T01:19:51.901770+0000","last_clean":"2024-09-11T01:19:51.901770+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:51.901770+0000","last_undegraded":"2024-09-11T01:19:51.901770+0000","last_fullsized":"2024-09-11T01:19:51.901770+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33182179,"num_objects":8102,"num_object_clones":0,"num_object_copies":16204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8974,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12116","reported_seq":12306,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008942+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:52.008942+0000","last_peered":"2024-09-11T01:19:52.008942+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:52.008942+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8604,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4007},{"shards":"0,2","objects":2104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12459","reported_seq":12660,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.989862+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:51.989862+0000","last_peered":"2024-09-11T01:19:51.989862+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:51.989862+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34214854,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9212,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12227","reported_seq":12342,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.942671+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:51.942671+0000","last_peered":"2024-09-11T01:19:51.942671+0000","last_clean":"2024-09-11T01:19:51.942671+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:51.942671+0000","last_undegraded":"2024-09-11T01:19:51.942671+0000","last_fullsized":"2024-09-11T01:19:51.942671+0000","mapping_epoch":31,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33574912,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9180,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12189","reported_seq":20528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.492570+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:54.492570+0000","last_peered":"2024-09-11T01:19:54.492570+0000","last_clean":"2024-09-11T01:19:54.492570+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:54.492570+0000","last_undegraded":"2024-09-11T01:19:54.492570+0000","last_fullsized":"2024-09-11T01:19:54.492570+0000","mapping_epoch":40,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33014243,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10056,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12287","reported_seq":12468,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.007362+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:52.007362+0000","last_peered":"2024-09-11T01:19:52.007362+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:52.007362+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8651,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":2119}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12141","reported_seq":12275,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.534479+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:55.534479+0000","last_peered":"2024-09-11T01:19:55.534479+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:55.534479+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32760291,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9910,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1958},{"shards":"2","objects":3995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12413","reported_seq":12546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.537190+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:55.537190+0000","last_peered":"2024-09-11T01:19:55.537190+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:55.537190+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33378304,"num_objects":8149,"num_object_clones":0,"num_object_copies":16298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10050,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2048},{"shards":"2","objects":3994}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12167","reported_seq":17334,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:52.012628+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:52.012628+0000","last_peered":"2024-09-11T01:19:52.012628+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:52.012628+0000","last_undegraded":"2024-09-11T01:19:52.012628+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8857,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2520,"num_bytes_recovered":10317824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6540},{"shards":"0,2","objects":1645}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12346","reported_seq":12542,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.936973+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:51.936973+0000","last_peered":"2024-09-11T01:19:51.936973+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:51.936973+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8739,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4141},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12326","reported_seq":12527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.897971+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:51.897971+0000","last_peered":"2024-09-11T01:19:51.897971+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:51.897971+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12104","reported_seq":12269,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.922792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:51.922792+0000","last_peered":"2024-09-11T01:19:51.922792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:51.922792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32908230,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8761,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533795574,"num_objects":130331,"num_object_clones":0,"num_object_copies":260662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41052,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":130331,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154910,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6937,"num_bytes_recovered":27210211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1178357760,"data_stored":1178201483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676384,"kb_used_data":392016,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93695456,"statfs":{"total":96636764160,"available":95944146944,"internally_reserved":0,"allocated":401424384,"data_stored":401153007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,0,2,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790008,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608008,"kb_used_data":301432,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93763832,"statfs":{"total":96636764160,"available":96014163968,"internally_reserved":0,"allocated":308666368,"data_stored":308351171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820720,"kb_used_data":460056,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93551120,"statfs":{"total":96636764160,"available":95796346880,"internally_reserved":0,"allocated":471097344,"data_stored":470825962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469729280,"data_stored":469693150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308297728,"data_stored":308217783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400330752,"data_stored":400290550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:57.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:58.446 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:58.447 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:58.472 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-11T01:19:56.532941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":533795593,"num_objects":130333,"num_object_clones":0,"num_object_copies":260666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41052,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":130333,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154912,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6938,"num_bytes_recovered":27210230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105112,"kb_used_data":1153504,"kb_used_omap":12,"kb_used_meta":951539,"kb_avail":281010408,"statfs":{"total":289910292480,"available":287754657792,"internally_reserved":0,"allocated":1181188096,"data_stored":1180330140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":974375943},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,6,4,6,0,0,6,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20537344,"num_objects":-5014,"num_object_clones":0,"num_object_copies":-10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-239,"num_objects_unfound":0,"num_objects_dirty":-5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":978944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007362"},"pg_stats":[{"pgid":"2.f","version":"41'12102","reported_seq":12251,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008800+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:52.008800+0000","last_peered":"2024-09-11T01:19:52.008800+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:52.008800+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33325056,"num_objects":8136,"num_object_clones":0,"num_object_copies":16272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12102,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4053},{"shards":"0,2","objects":2140}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12383","reported_seq":12459,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:55.524304+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:55.524304+0000","last_peered":"2024-09-11T01:19:55.524304+0000","last_clean":"2024-09-11T01:19:55.524304+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:55.524304+0000","last_undegraded":"2024-09-11T01:19:55.524304+0000","last_fullsized":"2024-09-11T01:19:55.524304+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33338310,"num_objects":8141,"num_object_clones":0,"num_object_copies":16282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12099","reported_seq":12231,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.496662+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:55.496662+0000","last_peered":"2024-09-11T01:19:55.496662+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:55.496662+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546840,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10160,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":3968}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12312","reported_seq":12353,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:55.502626+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:55.502626+0000","last_peered":"2024-09-11T01:19:55.502626+0000","last_clean":"2024-09-11T01:19:55.502626+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:55.502626+0000","last_undegraded":"2024-09-11T01:19:55.502626+0000","last_fullsized":"2024-09-11T01:19:55.502626+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32972800,"num_objects":8050,"num_object_clones":0,"num_object_copies":16100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12312,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12176","reported_seq":12293,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.901770+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:51.901770+0000","last_peered":"2024-09-11T01:19:51.901770+0000","last_clean":"2024-09-11T01:19:51.901770+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:51.901770+0000","last_undegraded":"2024-09-11T01:19:51.901770+0000","last_fullsized":"2024-09-11T01:19:51.901770+0000","mapping_epoch":31,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33182179,"num_objects":8102,"num_object_clones":0,"num_object_copies":16204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8974,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12116","reported_seq":12306,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.008942+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:52.008942+0000","last_peered":"2024-09-11T01:19:52.008942+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:52.008942+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33141219,"num_objects":8092,"num_object_clones":0,"num_object_copies":16184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8604,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4007},{"shards":"0,2","objects":2104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12459","reported_seq":12660,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.989862+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:51.989862+0000","last_peered":"2024-09-11T01:19:51.989862+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:51.989862+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34214854,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9212,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4108},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12227","reported_seq":12342,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.942671+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:51.942671+0000","last_peered":"2024-09-11T01:19:51.942671+0000","last_clean":"2024-09-11T01:19:51.942671+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:51.942671+0000","last_undegraded":"2024-09-11T01:19:51.942671+0000","last_fullsized":"2024-09-11T01:19:51.942671+0000","mapping_epoch":31,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33574912,"num_objects":8197,"num_object_clones":0,"num_object_copies":16394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9180,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12189","reported_seq":20528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.492570+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:54.492570+0000","last_peered":"2024-09-11T01:19:54.492570+0000","last_clean":"2024-09-11T01:19:54.492570+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:54.492570+0000","last_undegraded":"2024-09-11T01:19:54.492570+0000","last_fullsized":"2024-09-11T01:19:54.492570+0000","mapping_epoch":40,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33014243,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10056,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12287","reported_seq":12468,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.007362+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:52.007362+0000","last_peered":"2024-09-11T01:19:52.007362+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:52.007362+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34025472,"num_objects":8307,"num_object_clones":0,"num_object_copies":16614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8651,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4230},{"shards":"0,2","objects":2119}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12141","reported_seq":12275,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.534479+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:55.534479+0000","last_peered":"2024-09-11T01:19:55.534479+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:55.534479+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32760291,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9910,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1958},{"shards":"2","objects":3995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12413","reported_seq":12546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.537190+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:55.537190+0000","last_peered":"2024-09-11T01:19:55.537190+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:55.537190+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33378304,"num_objects":8149,"num_object_clones":0,"num_object_copies":16298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10050,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2048},{"shards":"2","objects":3994}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12167","reported_seq":17334,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:52.012628+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:52.012628+0000","last_peered":"2024-09-11T01:19:52.012628+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:52.012628+0000","last_undegraded":"2024-09-11T01:19:52.012628+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8857,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2520,"num_bytes_recovered":10317824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6540},{"shards":"0,2","objects":1645}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12346","reported_seq":12542,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.936973+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:51.936973+0000","last_peered":"2024-09-11T01:19:51.936973+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:51.936973+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8739,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4141},{"shards":"0,2","objects":2173}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12326","reported_seq":12527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.897971+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:51.897971+0000","last_peered":"2024-09-11T01:19:51.897971+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:51.897971+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33890304,"num_objects":8274,"num_object_clones":0,"num_object_copies":16548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8861,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4162},{"shards":"0,2","objects":2124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12104","reported_seq":12269,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.922792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:51.922792+0000","last_peered":"2024-09-11T01:19:51.922792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:51.922792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32908230,"num_objects":8036,"num_object_clones":0,"num_object_copies":16072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8761,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":533795574,"num_objects":130331,"num_object_clones":0,"num_object_copies":260662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41052,"num_objects_misplaced":1645,"num_objects_unfound":0,"num_objects_dirty":130331,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":154910,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6937,"num_bytes_recovered":27210211,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1178357760,"data_stored":1178201483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676384,"kb_used_data":392016,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93695456,"statfs":{"total":96636764160,"available":95944146944,"internally_reserved":0,"allocated":401424384,"data_stored":401153007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,0,2,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790008,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608008,"kb_used_data":301432,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93763832,"statfs":{"total":96636764160,"available":96014163968,"internally_reserved":0,"allocated":308666368,"data_stored":308351171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820720,"kb_used_data":460056,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93551120,"statfs":{"total":96636764160,"available":95796346880,"internally_reserved":0,"allocated":471097344,"data_stored":470825962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,1,0,0,6],"upper_bound":1024},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469729280,"data_stored":469693150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308297728,"data_stored":308217783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400330752,"data_stored":400290550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12102", 'reported_seq': 12251, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:52.008800+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:19:52.008800+0000', 'last_peered': '2024-09-11T01:19:52.008800+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:19:52.008800+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33325056, 'num_objects': 8136, 'num_object_clones': 0, 'num_object_copies': 16272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4083, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12102, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4053}, {'shards': '0,2', 'objects': 2140}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12099", 'reported_seq': 12231, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:55.496662+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:19:55.496662+0000', 'last_peered': '2024-09-11T01:19:55.496662+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:19:55.496662+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2000", 'ondisk_log_start': "32'2000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32546840, 'num_objects': 7947, 'num_object_clones': 0, 'num_object_copies': 15894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3980, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7947, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10160, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1930}, {'shards': '2', 'objects': 3968}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12116", 'reported_seq': 12306, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:52.008942+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:19:52.008942+0000', 'last_peered': '2024-09-11T01:19:52.008942+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:19:52.008942+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33141219, 'num_objects': 8092, 'num_object_clones': 0, 'num_object_copies': 16184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4086, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8092, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8604, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4007}, {'shards': '0,2', 'objects': 2104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'12459", 'reported_seq': 12660, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.989862+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:19:51.989862+0000', 'last_peered': '2024-09-11T01:19:51.989862+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:19:51.989862+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34214854, 'num_objects': 8355, 'num_object_clones': 0, 'num_object_copies': 16710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4247, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8355, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9212, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4108}, {'shards': '0,2', 'objects': 2233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12287", 'reported_seq': 12468, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:52.007362+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:19:52.007362+0000', 'last_peered': '2024-09-11T01:19:52.007362+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:19:52.007362+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2200", 'ondisk_log_start': "32'2200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34025472, 'num_objects': 8307, 'num_object_clones': 0, 'num_object_copies': 16614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8307, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8651, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4230}, {'shards': '0,2', 'objects': 2119}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12141", 'reported_seq': 12275, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:55.534479+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:19:55.534479+0000', 'last_peered': '2024-09-11T01:19:55.534479+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:19:55.534479+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32760291, 'num_objects': 7999, 'num_object_clones': 0, 'num_object_copies': 15998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4005, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7999, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9910, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 23, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1958}, {'shards': '2', 'objects': 3995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12413", 'reported_seq': 12546, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:55.537190+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:19:55.537190+0000', 'last_peered': '2024-09-11T01:19:55.537190+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:19:55.537190+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33378304, 'num_objects': 8149, 'num_object_clones': 0, 'num_object_copies': 16298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10050, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2048}, {'shards': '2', 'objects': 3994}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'12167", 'reported_seq': 17334, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:19:52.012628+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:19:52.012628+0000', 'last_peered': '2024-09-11T01:19:52.012628+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:19:52.012628+0000', 'last_undegraded': '2024-09-11T01:19:52.012628+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33525760, 'num_objects': 8185, 'num_object_clones': 0, 'num_object_copies': 16370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1645, 'num_objects_unfound': 0, 'num_objects_dirty': 8185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8857, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2520, 'num_bytes_recovered': 10317824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6540}, {'shards': '0,2', 'objects': 1645}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:19:58.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'12346", 'reported_seq': 12542, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.936973+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:19:51.936973+0000', 'last_peered': '2024-09-11T01:19:51.936973+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:19:51.936973+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33996800, 'num_objects': 8300, 'num_object_clones': 0, 'num_object_copies': 16600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4162, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8739, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4141}, {'shards': '0,2', 'objects': 2173}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.476 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:58.476 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'12326", 'reported_seq': 12527, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.897971+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:19:51.897971+0000', 'last_peered': '2024-09-11T01:19:51.897971+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:19:51.897971+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33890304, 'num_objects': 8274, 'num_object_clones': 0, 'num_object_copies': 16548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8861, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4162}, {'shards': '0,2', 'objects': 2124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.476 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:58.476 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12104", 'reported_seq': 12269, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:51.922792+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:19:51.922792+0000', 'last_peered': '2024-09-11T01:19:51.922792+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:19:51.922792+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2100", 'ondisk_log_start': "32'2100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32908230, 'num_objects': 8036, 'num_object_clones': 0, 'num_object_copies': 16072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8036, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8761, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 17, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3899}, {'shards': '0,2', 'objects': 2125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:58.476 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:59.203 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:59.208 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41019/258010 objects degraded (15.898%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":399,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129005,"data_bytes":528356105,"bytes_used":2153717760,"bytes_avail":287756574720,"bytes_total":289910292480,"degraded_objects":41019,"degraded_total":258010,"degraded_ratio":0.15898220999186077,"misplaced_objects":1490,"misplaced_total":258010,"misplaced_ratio":0.0057749699624045584,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":134393,"recovering_keys_per_sec":0,"num_objects_recovered":468,"num_bytes_recovered":1613824,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":471},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:59.238 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:59.238 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:59.806 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:19:59.806 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:19:59.836 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-11T01:19:58.534633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":528356105,"num_objects":129005,"num_object_clones":0,"num_object_copies":258010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41019,"num_objects_misplaced":1490,"num_objects_unfound":0,"num_objects_dirty":129005,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":156240,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7126,"num_bytes_recovered":27845110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160678,"ondisk_log_size":160678,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2103240,"kb_used_data":1147480,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":281012280,"statfs":{"total":289910292480,"available":287756574720,"internally_reserved":0,"allocated":1175019520,"data_stored":1174161564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":978635783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,5,3,5,0,0,0,0,0,10],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23171072,"num_objects":-5657,"num_object_clones":0,"num_object_copies":-11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-74,"num_objects_misplaced":-394,"num_objects_unfound":0,"num_objects_dirty":-5657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5657,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":1613824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008171"},"pg_stats":[{"pgid":"2.f","version":"41'12234","reported_seq":12387,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:56.966625+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:19:56.966625+0000","last_peered":"2024-09-11T01:19:56.966625+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:19:56.966625+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32784384,"num_objects":8004,"num_object_clones":0,"num_object_copies":16008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12234,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3923},{"shards":"0,2","objects":2008}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12383","reported_seq":12459,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:55.524304+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:19:55.524304+0000","last_peered":"2024-09-11T01:19:55.524304+0000","last_clean":"2024-09-11T01:19:55.524304+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:19:55.524304+0000","last_undegraded":"2024-09-11T01:19:55.524304+0000","last_fullsized":"2024-09-11T01:19:55.524304+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33338310,"num_objects":8141,"num_object_clones":0,"num_object_copies":16282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12099","reported_seq":12231,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.496662+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:19:55.496662+0000","last_peered":"2024-09-11T01:19:55.496662+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:19:55.496662+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2000","ondisk_log_start":"32'2000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32546840,"num_objects":7947,"num_object_clones":0,"num_object_copies":15894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10160,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1930},{"shards":"2","objects":3968}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12312","reported_seq":12353,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:55.502626+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:19:55.502626+0000","last_peered":"2024-09-11T01:19:55.502626+0000","last_clean":"2024-09-11T01:19:55.502626+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:19:55.502626+0000","last_undegraded":"2024-09-11T01:19:55.502626+0000","last_fullsized":"2024-09-11T01:19:55.502626+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32972800,"num_objects":8050,"num_object_clones":0,"num_object_copies":16100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12312,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12307","reported_seq":12424,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:56.892378+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:19:56.892378+0000","last_peered":"2024-09-11T01:19:56.892378+0000","last_clean":"2024-09-11T01:19:56.892378+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:19:56.892378+0000","last_undegraded":"2024-09-11T01:19:56.892378+0000","last_fullsized":"2024-09-11T01:19:56.892378+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32645603,"num_objects":7971,"num_object_clones":0,"num_object_copies":15942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9105,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12243","reported_seq":12447,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.008324+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:19:57.008324+0000","last_peered":"2024-09-11T01:19:57.008324+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:19:57.008324+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32621027,"num_objects":7965,"num_object_clones":0,"num_object_copies":15930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7965,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8731,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3887},{"shards":"0,2","objects":1977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12608","reported_seq":12815,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.042798+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:19:57.042798+0000","last_peered":"2024-09-11T01:19:57.042798+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:19:57.042798+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33604550,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9361,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3962},{"shards":"0,2","objects":2084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12345","reported_seq":12460,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:56.993720+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:19:56.993720+0000","last_peered":"2024-09-11T01:19:56.993720+0000","last_clean":"2024-09-11T01:19:56.993720+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:19:56.993720+0000","last_undegraded":"2024-09-11T01:19:56.993720+0000","last_fullsized":"2024-09-11T01:19:56.993720+0000","mapping_epoch":31,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33091584,"num_objects":8079,"num_object_clones":0,"num_object_copies":16158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9298,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12189","reported_seq":20528,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.492570+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:54.492570+0000","last_peered":"2024-09-11T01:19:54.492570+0000","last_clean":"2024-09-11T01:19:54.492570+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:54.492570+0000","last_undegraded":"2024-09-11T01:19:54.492570+0000","last_fullsized":"2024-09-11T01:19:54.492570+0000","mapping_epoch":40,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33014243,"num_objects":8061,"num_object_clones":0,"num_object_copies":16122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10056,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12419","reported_seq":12604,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.015043+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:19:57.015043+0000","last_peered":"2024-09-11T01:19:57.015043+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:19:57.015043+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8783,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4100},{"shards":"0,2","objects":1987}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12141","reported_seq":12275,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.534479+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:19:55.534479+0000","last_peered":"2024-09-11T01:19:55.534479+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:19:55.534479+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2100","ondisk_log_start":"32'2100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32760291,"num_objects":7999,"num_object_clones":0,"num_object_copies":15998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9910,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1958},{"shards":"2","objects":3995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12413","reported_seq":12546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:55.537190+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:19:55.537190+0000","last_peered":"2024-09-11T01:19:55.537190+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:19:55.537190+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33378304,"num_objects":8149,"num_object_clones":0,"num_object_copies":16298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10050,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2048},{"shards":"2","objects":3994}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12305","reported_seq":17782,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:19:57.043217+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:19:57.043217+0000","last_peered":"2024-09-11T01:19:57.043217+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:19:57.043217+0000","last_undegraded":"2024-09-11T01:19:57.043217+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32960512,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1490,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8995,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2675,"num_bytes_recovered":10952704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6557},{"shards":"0,2","objects":1490}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12480","reported_seq":12685,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.060531+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:19:57.060531+0000","last_peered":"2024-09-11T01:19:57.060531+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:19:57.060531+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8873,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4012},{"shards":"0,2","objects":2039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12470","reported_seq":12689,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.048187+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:19:57.048187+0000","last_peered":"2024-09-11T01:19:57.048187+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:19:57.048187+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33300480,"num_objects":8130,"num_object_clones":0,"num_object_copies":16260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4105,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9005,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4027},{"shards":"0,2","objects":1980}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12227","reported_seq":12402,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.020095+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:19:57.020095+0000","last_peered":"2024-09-11T01:19:57.020095+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:19:57.020095+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404422,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8884,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3781},{"shards":"0,2","objects":2002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":528356086,"num_objects":129003,"num_object_clones":0,"num_object_copies":258006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41019,"num_objects_misplaced":1490,"num_objects_unfound":0,"num_objects_dirty":129003,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":156238,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7125,"num_bytes_recovered":27845091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1171853312,"data_stored":1171697035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676384,"kb_used_data":392016,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93695456,"statfs":{"total":96636764160,"available":95944146944,"internally_reserved":0,"allocated":401424384,"data_stored":401153007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,0,2,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790008,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608008,"kb_used_data":301432,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93763832,"statfs":{"total":96636764160,"available":96014163968,"internally_reserved":0,"allocated":308666368,"data_stored":308351171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818848,"kb_used_data":454032,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93552992,"statfs":{"total":96636764160,"available":95798263808,"internally_reserved":0,"allocated":464928768,"data_stored":464657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,0,0,0,0,0,0,8],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463224832,"data_stored":463188702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308297728,"data_stored":308217783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":400330752,"data_stored":400290550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:02.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:03.431 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:03.432 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:03.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-11T01:20:02.536024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519643913,"num_objects":126878,"num_object_clones":0,"num_object_copies":253756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41013,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":126878,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":158367,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7272,"num_bytes_recovered":28418550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2096344,"kb_used_data":1136416,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":281019176,"statfs":{"total":289910292480,"available":287763636224,"internally_reserved":0,"allocated":1163689984,"data_stored":1162832028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":982895623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,3,4,4,4,0,0,0,7,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25776128,"num_objects":-6293,"num_object_clones":0,"num_object_copies":-12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-66,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":-6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6293,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004953"},"pg_stats":[{"pgid":"2.f","version":"41'12370","reported_seq":12525,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.075581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:02.075581+0000","last_peered":"2024-09-11T01:20:02.075581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:02.075581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227328,"num_objects":7868,"num_object_clones":0,"num_object_copies":15736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12370,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":1872}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12504","reported_seq":12580,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:00.387084+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:00.387084+0000","last_peered":"2024-09-11T01:20:00.387084+0000","last_clean":"2024-09-11T01:20:00.387084+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:00.387084+0000","last_undegraded":"2024-09-11T01:20:00.387084+0000","last_fullsized":"2024-09-11T01:20:00.387084+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32842694,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10602,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12217","reported_seq":12349,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.427663+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:00.427663+0000","last_peered":"2024-09-11T01:20:00.427663+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:00.427663+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32063512,"num_objects":7829,"num_object_clones":0,"num_object_copies":15658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7829,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10278,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1812},{"shards":"2","objects":3850}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12449","reported_seq":12490,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:00.565194+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:00.565194+0000","last_peered":"2024-09-11T01:20:00.565194+0000","last_clean":"2024-09-11T01:20:00.565194+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:00.565194+0000","last_undegraded":"2024-09-11T01:20:00.565194+0000","last_fullsized":"2024-09-11T01:20:00.565194+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32411648,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12449,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12426","reported_seq":12543,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.032093+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:02.032093+0000","last_peered":"2024-09-11T01:20:02.032093+0000","last_clean":"2024-09-11T01:20:02.032093+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:02.032093+0000","last_undegraded":"2024-09-11T01:20:02.032093+0000","last_fullsized":"2024-09-11T01:20:02.032093+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158179,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9224,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12363","reported_seq":12567,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.033079+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:02.033079+0000","last_peered":"2024-09-11T01:20:02.033079+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:02.033079+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129507,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8851,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":1857}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12749","reported_seq":12957,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:01.994059+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:01.994059+0000","last_peered":"2024-09-11T01:20:01.994059+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:01.994059+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33027014,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3822},{"shards":"0,2","objects":1943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12487","reported_seq":12602,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.039200+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:02.039200+0000","last_peered":"2024-09-11T01:20:02.039200+0000","last_clean":"2024-09-11T01:20:02.039200+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:02.039200+0000","last_undegraded":"2024-09-11T01:20:02.039200+0000","last_fullsized":"2024-09-11T01:20:02.039200+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32509952,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9440,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12308","reported_seq":20647,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.536849+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:59.536849+0000","last_peered":"2024-09-11T01:19:59.536849+0000","last_clean":"2024-09-11T01:19:59.536849+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:59.536849+0000","last_undegraded":"2024-09-11T01:19:59.536849+0000","last_fullsized":"2024-09-11T01:19:59.536849+0000","mapping_epoch":40,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526819,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10175,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12565","reported_seq":12752,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.031633+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:02.031633+0000","last_peered":"2024-09-11T01:20:02.031633+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:02.031633+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32886784,"num_objects":8029,"num_object_clones":0,"num_object_copies":16058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8929,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3955},{"shards":"0,2","objects":1841}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12268","reported_seq":12408,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.537712+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:00.537712+0000","last_peered":"2024-09-11T01:20:00.537712+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:00.537712+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32240099,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10037,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1831},{"shards":"2","objects":3870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12555","reported_seq":12687,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.571570+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:00.571570+0000","last_peered":"2024-09-11T01:20:00.571570+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:00.571570+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32796672,"num_objects":8007,"num_object_clones":0,"num_object_copies":16014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10192,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1906},{"shards":"2","objects":3852}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12434","reported_seq":18191,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:02.099878+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:02.099878+0000","last_peered":"2024-09-11T01:20:02.099878+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:02.099878+0000","last_undegraded":"2024-09-11T01:20:02.099878+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9124,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":11526144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":1350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12608","reported_seq":12814,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:01.998882+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:01.998882+0000","last_peered":"2024-09-11T01:20:01.998882+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:01.998882+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9001,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12612","reported_seq":12833,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.014940+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:02.014940+0000","last_peered":"2024-09-11T01:20:02.014940+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:02.014940+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9147,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3886},{"shards":"0,2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12387","reported_seq":12561,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.052650+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:02.052650+0000","last_peered":"2024-09-11T01:20:02.052650+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:02.052650+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31749062,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9044,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":519643894,"num_objects":126876,"num_object_clones":0,"num_object_copies":253752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41013,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":126876,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":158365,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7271,"num_bytes_recovered":28418531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1160413184,"data_stored":1160256907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676420,"kb_used_data":387892,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93695420,"statfs":{"total":96636764160,"available":95944110080,"internally_reserved":0,"allocated":397201408,"data_stored":396930031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,0,2,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790009,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615932,"kb_used_data":301036,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93755908,"statfs":{"total":96636764160,"available":96006049792,"internally_reserved":0,"allocated":308260864,"data_stored":307945667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":322432687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803992,"kb_used_data":447488,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93567848,"statfs":{"total":96636764160,"available":95813476352,"internally_reserved":0,"allocated":458227712,"data_stored":457956330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,1,1,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":456544256,"data_stored":456508126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307822592,"data_stored":307742647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396046336,"data_stored":396006134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:03.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:04.014 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:04.015 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:04.045 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-11T01:20:02.536024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519643913,"num_objects":126878,"num_object_clones":0,"num_object_copies":253756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41013,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":126878,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":158367,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7272,"num_bytes_recovered":28418550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2096344,"kb_used_data":1136416,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":281019176,"statfs":{"total":289910292480,"available":287763636224,"internally_reserved":0,"allocated":1163689984,"data_stored":1162832028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":982895623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,3,4,4,4,0,0,0,7,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25776128,"num_objects":-6293,"num_object_clones":0,"num_object_copies":-12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-66,"num_objects_misplaced":-414,"num_objects_unfound":0,"num_objects_dirty":-6293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6293,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":480,"num_bytes_recovered":1695744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004953"},"pg_stats":[{"pgid":"2.f","version":"41'12370","reported_seq":12525,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.075581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:02.075581+0000","last_peered":"2024-09-11T01:20:02.075581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:02.075581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227328,"num_objects":7868,"num_object_clones":0,"num_object_copies":15736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12370,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":1872}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12504","reported_seq":12580,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:00.387084+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:00.387084+0000","last_peered":"2024-09-11T01:20:00.387084+0000","last_clean":"2024-09-11T01:20:00.387084+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:00.387084+0000","last_undegraded":"2024-09-11T01:20:00.387084+0000","last_fullsized":"2024-09-11T01:20:00.387084+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32842694,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10602,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12217","reported_seq":12349,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.427663+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:00.427663+0000","last_peered":"2024-09-11T01:20:00.427663+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:00.427663+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32063512,"num_objects":7829,"num_object_clones":0,"num_object_copies":15658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7829,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10278,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1812},{"shards":"2","objects":3850}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12449","reported_seq":12490,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:00.565194+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:00.565194+0000","last_peered":"2024-09-11T01:20:00.565194+0000","last_clean":"2024-09-11T01:20:00.565194+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:00.565194+0000","last_undegraded":"2024-09-11T01:20:00.565194+0000","last_fullsized":"2024-09-11T01:20:00.565194+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32411648,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12449,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12426","reported_seq":12543,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.032093+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:02.032093+0000","last_peered":"2024-09-11T01:20:02.032093+0000","last_clean":"2024-09-11T01:20:02.032093+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:02.032093+0000","last_undegraded":"2024-09-11T01:20:02.032093+0000","last_fullsized":"2024-09-11T01:20:02.032093+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158179,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9224,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12363","reported_seq":12567,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.033079+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:02.033079+0000","last_peered":"2024-09-11T01:20:02.033079+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:02.033079+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129507,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8851,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":1857}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12749","reported_seq":12957,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:01.994059+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:01.994059+0000","last_peered":"2024-09-11T01:20:01.994059+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:01.994059+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33027014,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3822},{"shards":"0,2","objects":1943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12487","reported_seq":12602,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.039200+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:02.039200+0000","last_peered":"2024-09-11T01:20:02.039200+0000","last_clean":"2024-09-11T01:20:02.039200+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:02.039200+0000","last_undegraded":"2024-09-11T01:20:02.039200+0000","last_fullsized":"2024-09-11T01:20:02.039200+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32509952,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9440,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12308","reported_seq":20647,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.536849+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:59.536849+0000","last_peered":"2024-09-11T01:19:59.536849+0000","last_clean":"2024-09-11T01:19:59.536849+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:59.536849+0000","last_undegraded":"2024-09-11T01:19:59.536849+0000","last_fullsized":"2024-09-11T01:19:59.536849+0000","mapping_epoch":40,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526819,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10175,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12565","reported_seq":12752,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.031633+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:02.031633+0000","last_peered":"2024-09-11T01:20:02.031633+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:02.031633+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32886784,"num_objects":8029,"num_object_clones":0,"num_object_copies":16058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8929,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3955},{"shards":"0,2","objects":1841}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12268","reported_seq":12408,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.537712+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:00.537712+0000","last_peered":"2024-09-11T01:20:00.537712+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:00.537712+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32240099,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10037,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1831},{"shards":"2","objects":3870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12555","reported_seq":12687,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.571570+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:00.571570+0000","last_peered":"2024-09-11T01:20:00.571570+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:00.571570+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32796672,"num_objects":8007,"num_object_clones":0,"num_object_copies":16014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10192,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1906},{"shards":"2","objects":3852}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12434","reported_seq":18191,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:02.099878+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:02.099878+0000","last_peered":"2024-09-11T01:20:02.099878+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:02.099878+0000","last_undegraded":"2024-09-11T01:20:02.099878+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9124,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":11526144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":1350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12608","reported_seq":12814,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:01.998882+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:01.998882+0000","last_peered":"2024-09-11T01:20:01.998882+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:01.998882+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9001,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12612","reported_seq":12833,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.014940+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:02.014940+0000","last_peered":"2024-09-11T01:20:02.014940+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:02.014940+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9147,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3886},{"shards":"0,2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12387","reported_seq":12561,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.052650+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:02.052650+0000","last_peered":"2024-09-11T01:20:02.052650+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:02.052650+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31749062,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9044,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":519643894,"num_objects":126876,"num_object_clones":0,"num_object_copies":253752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41013,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":126876,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":158365,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7271,"num_bytes_recovered":28418531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1160413184,"data_stored":1160256907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676420,"kb_used_data":387892,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93695420,"statfs":{"total":96636764160,"available":95944110080,"internally_reserved":0,"allocated":397201408,"data_stored":396930031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,0,2,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790009,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615932,"kb_used_data":301036,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93755908,"statfs":{"total":96636764160,"available":96006049792,"internally_reserved":0,"allocated":308260864,"data_stored":307945667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":322432687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803992,"kb_used_data":447488,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93567848,"statfs":{"total":96636764160,"available":95813476352,"internally_reserved":0,"allocated":458227712,"data_stored":457956330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,1,1,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":456544256,"data_stored":456508126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307822592,"data_stored":307742647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396046336,"data_stored":396006134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:04.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:04.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12370", 'reported_seq': 12525, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.075581+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:02.075581+0000', 'last_peered': '2024-09-11T01:20:02.075581+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:02.075581+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32227328, 'num_objects': 7868, 'num_object_clones': 0, 'num_object_copies': 15736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4080, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12370, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3788}, {'shards': '0,2', 'objects': 1872}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:04.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12217", 'reported_seq': 12349, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:00.427663+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:00.427663+0000', 'last_peered': '2024-09-11T01:20:00.427663+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:00.427663+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2200", 'ondisk_log_start': "32'2200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32063512, 'num_objects': 7829, 'num_object_clones': 0, 'num_object_copies': 15658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3980, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7829, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10278, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1812}, {'shards': '2', 'objects': 3850}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:04.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:04.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12363", 'reported_seq': 12567, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.033079+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:02.033079+0000', 'last_peered': '2024-09-11T01:20:02.033079+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:02.033079+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32129507, 'num_objects': 7845, 'num_object_clones': 0, 'num_object_copies': 15690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4079, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7845, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8851, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3767}, {'shards': '0,2', 'objects': 1857}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:04.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'12749", 'reported_seq': 12957, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:01.994059+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:01.994059+0000', 'last_peered': '2024-09-11T01:20:01.994059+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:01.994059+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33027014, 'num_objects': 8065, 'num_object_clones': 0, 'num_object_copies': 16130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4243, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8065, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9502, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3822}, {'shards': '0,2', 'objects': 1943}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:04.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12565", 'reported_seq': 12752, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.031633+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:02.031633+0000', 'last_peered': '2024-09-11T01:20:02.031633+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:02.031633+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2500", 'ondisk_log_start': "32'2500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32886784, 'num_objects': 8029, 'num_object_clones': 0, 'num_object_copies': 16058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4076, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8929, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3955}, {'shards': '0,2', 'objects': 1841}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:04.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12268", 'reported_seq': 12408, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:00.537712+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:00.537712+0000', 'last_peered': '2024-09-11T01:20:00.537712+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:00.537712+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2200", 'ondisk_log_start': "32'2200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32240099, 'num_objects': 7872, 'num_object_clones': 0, 'num_object_copies': 15744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7872, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10037, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1831}, {'shards': '2', 'objects': 3870}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:04.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:04.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12555", 'reported_seq': 12687, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:00.571570+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:00.571570+0000', 'last_peered': '2024-09-11T01:20:00.571570+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:00.571570+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2500", 'ondisk_log_start': "32'2500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32796672, 'num_objects': 8007, 'num_object_clones': 0, 'num_object_copies': 16014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10192, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1906}, {'shards': '2', 'objects': 3852}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:04.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:04.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'12434", 'reported_seq': 18191, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:02.099878+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:02.099878+0000', 'last_peered': '2024-09-11T01:20:02.099878+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:02.099878+0000', 'last_undegraded': '2024-09-11T01:20:02.099878+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32432128, 'num_objects': 7918, 'num_object_clones': 0, 'num_object_copies': 15836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1350, 'num_objects_unfound': 0, 'num_objects_dirty': 7918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9124, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2815, 'num_bytes_recovered': 11526144, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6568}, {'shards': '0,2', 'objects': 1350}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:04.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'12608", 'reported_seq': 12814, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:01.998882+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:01.998882+0000', 'last_peered': '2024-09-11T01:20:01.998882+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:01.998882+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32923648, 'num_objects': 8038, 'num_object_clones': 0, 'num_object_copies': 16076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9001, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 38, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3884}, {'shards': '0,2', 'objects': 1911}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:04.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'12612", 'reported_seq': 12833, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.014940+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:02.014940+0000', 'last_peered': '2024-09-11T01:20:02.014940+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:02.014940+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32718848, 'num_objects': 7988, 'num_object_clones': 0, 'num_object_copies': 15976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9147, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3886}, {'shards': '0,2', 'objects': 1838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:04.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12387", 'reported_seq': 12561, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.052650+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:02.052650+0000', 'last_peered': '2024-09-11T01:20:02.052650+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:02.052650+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31749062, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4134, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9044, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3621}, {'shards': '0,2', 'objects': 1842}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:04.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:04.734 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:04.735 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41013/253756 objects degraded (16.162%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":405,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":126878,"data_bytes":519643913,"bytes_used":2146656256,"bytes_avail":287763636224,"bytes_total":289910292480,"degraded_objects":41013,"degraded_total":253756,"degraded_ratio":0.16162376456123206,"misplaced_objects":1350,"misplaced_total":253756,"misplaced_ratio":0.0053200712495468086,"recovering_objects_per_sec":39,"recovering_bytes_per_sec":141253,"recovering_keys_per_sec":0,"num_objects_recovered":480,"num_bytes_recovered":1695744,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":524},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:04.761 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:04.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:05.354 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:05.356 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:05.376 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-11T01:20:04.536861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519643913,"num_objects":126878,"num_object_clones":0,"num_object_copies":253756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41013,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":126878,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":158367,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7272,"num_bytes_recovered":28418550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160705,"ondisk_log_size":160705,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2096344,"kb_used_data":1136416,"kb_used_omap":12,"kb_used_meta":959859,"kb_avail":281019176,"statfs":{"total":289910292480,"available":287763636224,"internally_reserved":0,"allocated":1163689984,"data_stored":1162832028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":982895623},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,3,4,4,4,0,0,0,7,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17469440,"num_objects":-4265,"num_object_clones":0,"num_object_copies":-8530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-41,"num_objects_misplaced":-295,"num_objects_unfound":0,"num_objects_dirty":-4265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4265,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":1208320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005295"},"pg_stats":[{"pgid":"2.f","version":"41'12370","reported_seq":12525,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.075581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:02.075581+0000","last_peered":"2024-09-11T01:20:02.075581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:02.075581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32227328,"num_objects":7868,"num_object_clones":0,"num_object_copies":15736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12370,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3788},{"shards":"0,2","objects":1872}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12504","reported_seq":12580,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:00.387084+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:00.387084+0000","last_peered":"2024-09-11T01:20:00.387084+0000","last_clean":"2024-09-11T01:20:00.387084+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:00.387084+0000","last_undegraded":"2024-09-11T01:20:00.387084+0000","last_fullsized":"2024-09-11T01:20:00.387084+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32842694,"num_objects":8020,"num_object_clones":0,"num_object_copies":16040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8020,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10602,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12217","reported_seq":12349,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.427663+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:00.427663+0000","last_peered":"2024-09-11T01:20:00.427663+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:00.427663+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32063512,"num_objects":7829,"num_object_clones":0,"num_object_copies":15658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7829,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10278,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1812},{"shards":"2","objects":3850}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12449","reported_seq":12490,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:00.565194+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:00.565194+0000","last_peered":"2024-09-11T01:20:00.565194+0000","last_clean":"2024-09-11T01:20:00.565194+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:00.565194+0000","last_undegraded":"2024-09-11T01:20:00.565194+0000","last_fullsized":"2024-09-11T01:20:00.565194+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32411648,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12449,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12426","reported_seq":12543,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.032093+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:02.032093+0000","last_peered":"2024-09-11T01:20:02.032093+0000","last_clean":"2024-09-11T01:20:02.032093+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:02.032093+0000","last_undegraded":"2024-09-11T01:20:02.032093+0000","last_fullsized":"2024-09-11T01:20:02.032093+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158179,"num_objects":7852,"num_object_clones":0,"num_object_copies":15704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7852,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9224,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12363","reported_seq":12567,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.033079+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:02.033079+0000","last_peered":"2024-09-11T01:20:02.033079+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:02.033079+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129507,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8851,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3767},{"shards":"0,2","objects":1857}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12749","reported_seq":12957,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:01.994059+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:01.994059+0000","last_peered":"2024-09-11T01:20:01.994059+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:01.994059+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33027014,"num_objects":8065,"num_object_clones":0,"num_object_copies":16130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3822},{"shards":"0,2","objects":1943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12487","reported_seq":12602,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.039200+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:02.039200+0000","last_peered":"2024-09-11T01:20:02.039200+0000","last_clean":"2024-09-11T01:20:02.039200+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:02.039200+0000","last_undegraded":"2024-09-11T01:20:02.039200+0000","last_fullsized":"2024-09-11T01:20:02.039200+0000","mapping_epoch":31,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32509952,"num_objects":7937,"num_object_clones":0,"num_object_copies":15874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9440,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12308","reported_seq":20647,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.536849+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:19:59.536849+0000","last_peered":"2024-09-11T01:19:59.536849+0000","last_clean":"2024-09-11T01:19:59.536849+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:19:59.536849+0000","last_undegraded":"2024-09-11T01:19:59.536849+0000","last_fullsized":"2024-09-11T01:19:59.536849+0000","mapping_epoch":40,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32526819,"num_objects":7942,"num_object_clones":0,"num_object_copies":15884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7942,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10175,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12565","reported_seq":12752,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.031633+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:02.031633+0000","last_peered":"2024-09-11T01:20:02.031633+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:02.031633+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32886784,"num_objects":8029,"num_object_clones":0,"num_object_copies":16058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8929,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3955},{"shards":"0,2","objects":1841}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12268","reported_seq":12408,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.537712+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:00.537712+0000","last_peered":"2024-09-11T01:20:00.537712+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:00.537712+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2200","ondisk_log_start":"32'2200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32240099,"num_objects":7872,"num_object_clones":0,"num_object_copies":15744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10037,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1831},{"shards":"2","objects":3870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12555","reported_seq":12687,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:00.571570+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:00.571570+0000","last_peered":"2024-09-11T01:20:00.571570+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:00.571570+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32796672,"num_objects":8007,"num_object_clones":0,"num_object_copies":16014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10192,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1906},{"shards":"2","objects":3852}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12434","reported_seq":18191,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:02.099878+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:02.099878+0000","last_peered":"2024-09-11T01:20:02.099878+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:02.099878+0000","last_undegraded":"2024-09-11T01:20:02.099878+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32432128,"num_objects":7918,"num_object_clones":0,"num_object_copies":15836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":7918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9124,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2815,"num_bytes_recovered":11526144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":1350}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12608","reported_seq":12814,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:01.998882+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:01.998882+0000","last_peered":"2024-09-11T01:20:01.998882+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:01.998882+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9001,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3884},{"shards":"0,2","objects":1911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12612","reported_seq":12833,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.014940+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:02.014940+0000","last_peered":"2024-09-11T01:20:02.014940+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:02.014940+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32718848,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9147,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3886},{"shards":"0,2","objects":1838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12387","reported_seq":12561,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.052650+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:02.052650+0000","last_peered":"2024-09-11T01:20:02.052650+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:02.052650+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31749062,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9044,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3621},{"shards":"0,2","objects":1842}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":519643894,"num_objects":126876,"num_object_clones":0,"num_object_copies":253752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41013,"num_objects_misplaced":1350,"num_objects_unfound":0,"num_objects_dirty":126876,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":158365,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7271,"num_bytes_recovered":28418531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1160413184,"data_stored":1160256907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676420,"kb_used_data":387892,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93695420,"statfs":{"total":96636764160,"available":95944110080,"internally_reserved":0,"allocated":397201408,"data_stored":396930031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,0,2,0,0,0,0,0,2],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790009,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615932,"kb_used_data":301036,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93755908,"statfs":{"total":96636764160,"available":96006049792,"internally_reserved":0,"allocated":308260864,"data_stored":307945667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":322432687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803992,"kb_used_data":447488,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93567848,"statfs":{"total":96636764160,"available":95813476352,"internally_reserved":0,"allocated":458227712,"data_stored":457956330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,1,1,0,0,0,7],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":456544256,"data_stored":456508126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307822592,"data_stored":307742647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396046336,"data_stored":396006134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:08.379 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:08.931 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:08.931 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:08.962 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-11T01:20:08.538600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510808841,"num_objects":124721,"num_object_clones":0,"num_object_copies":249442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40979,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":124721,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":160524,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7453,"num_bytes_recovered":29020662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079712,"kb_used_data":1123940,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":281035808,"statfs":{"total":289910292480,"available":287780667392,"internally_reserved":0,"allocated":1150914560,"data_stored":1150056604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":978635783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,0,9,5,2,0,0,0,2,0,0,0,2],"upper_bound":32768},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22986752,"num_objects":-5612,"num_object_clones":0,"num_object_copies":-11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-73,"num_objects_misplaced":-442,"num_objects_unfound":0,"num_objects_dirty":-5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":1810432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005658"},"pg_stats":[{"pgid":"2.f","version":"41'12515","reported_seq":12688,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.086374+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:07.086374+0000","last_peered":"2024-09-11T01:20:07.086374+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:07.086374+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12515,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":1727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12662","reported_seq":12738,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:05.557545+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:05.557545+0000","last_peered":"2024-09-11T01:20:05.557545+0000","last_clean":"2024-09-11T01:20:05.557545+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:05.557545+0000","last_undegraded":"2024-09-11T01:20:05.557545+0000","last_fullsized":"2024-09-11T01:20:05.557545+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32195526,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10760,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12346","reported_seq":12478,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.414048+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:05.414048+0000","last_peered":"2024-09-11T01:20:05.414048+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:05.414048+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31535128,"num_objects":7700,"num_object_clones":0,"num_object_copies":15400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10407,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1683},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12575","reported_seq":12616,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:05.537609+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:05.537609+0000","last_peered":"2024-09-11T01:20:05.537609+0000","last_clean":"2024-09-11T01:20:05.537609+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:05.537609+0000","last_undegraded":"2024-09-11T01:20:05.537609+0000","last_fullsized":"2024-09-11T01:20:05.537609+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31895552,"num_objects":7787,"num_object_clones":0,"num_object_copies":15574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12575,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12555","reported_seq":12672,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.021744+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:07.021744+0000","last_peered":"2024-09-11T01:20:07.021744+0000","last_clean":"2024-09-11T01:20:07.021744+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:07.021744+0000","last_undegraded":"2024-09-11T01:20:07.021744+0000","last_fullsized":"2024-09-11T01:20:07.021744+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629795,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9353,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12469","reported_seq":12679,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.037768+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:07.037768+0000","last_peered":"2024-09-11T01:20:07.037768+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:07.037768+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31695331,"num_objects":7739,"num_object_clones":0,"num_object_copies":15478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8957,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":1751}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12883","reported_seq":13106,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.086836+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:07.086836+0000","last_peered":"2024-09-11T01:20:07.086836+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:07.086836+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32478150,"num_objects":7931,"num_object_clones":0,"num_object_copies":15862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7931,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9636,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3695},{"shards":"0,2","objects":1809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12622","reported_seq":12737,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.072185+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:07.072185+0000","last_peered":"2024-09-11T01:20:07.072185+0000","last_clean":"2024-09-11T01:20:07.072185+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:07.072185+0000","last_undegraded":"2024-09-11T01:20:07.072185+0000","last_fullsized":"2024-09-11T01:20:07.072185+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31956992,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9575,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12458","reported_seq":20797,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:04.618289+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:04.618289+0000","last_peered":"2024-09-11T01:20:04.618289+0000","last_clean":"2024-09-11T01:20:04.618289+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:04.618289+0000","last_undegraded":"2024-09-11T01:20:04.618289+0000","last_fullsized":"2024-09-11T01:20:04.618289+0000","mapping_epoch":40,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31912419,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10325,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12705","reported_seq":12900,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.065539+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:07.065539+0000","last_peered":"2024-09-11T01:20:07.065539+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:07.065539+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3819},{"shards":"0,2","objects":1701}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12411","reported_seq":12551,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.504402+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:05.504402+0000","last_peered":"2024-09-11T01:20:05.504402+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:05.504402+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654371,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10180,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1688},{"shards":"2","objects":3727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12677","reported_seq":12810,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.556436+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:05.556436+0000","last_peered":"2024-09-11T01:20:05.556436+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:05.556436+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32296960,"num_objects":7885,"num_object_clones":0,"num_object_copies":15770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10314,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1784},{"shards":"2","objects":3730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12576","reported_seq":18627,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:07.107512+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:07.107512+0000","last_peered":"2024-09-11T01:20:07.107512+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:07.107512+0000","last_undegraded":"2024-09-11T01:20:07.107512+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":7776,"num_object_clones":0,"num_object_copies":15552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":7776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9266,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2962,"num_bytes_recovered":12128256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6573},{"shards":"0,2","objects":1203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12741","reported_seq":12951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.112702+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:07.112702+0000","last_peered":"2024-09-11T01:20:07.112702+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:07.112702+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9134,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3753},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12767","reported_seq":12996,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.099698+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:07.099698+0000","last_peered":"2024-09-11T01:20:07.099698+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:07.099698+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32083968,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9302,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12497","reported_seq":12680,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.040883+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:07.040883+0000","last_peered":"2024-09-11T01:20:07.040883+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:07.040883+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31298502,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9154,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3515},{"shards":"0,2","objects":1732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510808822,"num_objects":124719,"num_object_clones":0,"num_object_copies":249438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40979,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":124719,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":160522,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7452,"num_bytes_recovered":29020643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1149054976,"data_stored":1148898699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676268,"kb_used_data":383580,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93695572,"statfs":{"total":96636764160,"available":95944265728,"internally_reserved":0,"allocated":392785920,"data_stored":392514543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,0,0,0,0,0,0,0,0,0,2],"upper_bound":32768},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602900,"kb_used_data":300484,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93768940,"statfs":{"total":96636764160,"available":96019394560,"internally_reserved":0,"allocated":307695616,"data_stored":307380419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800544,"kb_used_data":439876,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93571296,"statfs":{"total":96636764160,"available":95817007104,"internally_reserved":0,"allocated":450433024,"data_stored":450161642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,4,4,2,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450019328,"data_stored":449983198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307326976,"data_stored":307247031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391708672,"data_stored":391668470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:08.964 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:09.527 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:09.528 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:09.551 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-11T01:20:08.538600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510808841,"num_objects":124721,"num_object_clones":0,"num_object_copies":249442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40979,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":124721,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":160524,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7453,"num_bytes_recovered":29020662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079712,"kb_used_data":1123940,"kb_used_omap":12,"kb_used_meta":955699,"kb_avail":281035808,"statfs":{"total":289910292480,"available":287780667392,"internally_reserved":0,"allocated":1150914560,"data_stored":1150056604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":978635783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,0,9,5,2,0,0,0,2,0,0,0,2],"upper_bound":32768},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22986752,"num_objects":-5612,"num_object_clones":0,"num_object_copies":-11224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-73,"num_objects_misplaced":-442,"num_objects_unfound":0,"num_objects_dirty":-5612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":515,"num_bytes_recovered":1810432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005658"},"pg_stats":[{"pgid":"2.f","version":"41'12515","reported_seq":12688,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.086374+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:07.086374+0000","last_peered":"2024-09-11T01:20:07.086374+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:07.086374+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12515,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":1727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12662","reported_seq":12738,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:05.557545+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:05.557545+0000","last_peered":"2024-09-11T01:20:05.557545+0000","last_clean":"2024-09-11T01:20:05.557545+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:05.557545+0000","last_undegraded":"2024-09-11T01:20:05.557545+0000","last_fullsized":"2024-09-11T01:20:05.557545+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32195526,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10760,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12346","reported_seq":12478,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.414048+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:05.414048+0000","last_peered":"2024-09-11T01:20:05.414048+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:05.414048+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31535128,"num_objects":7700,"num_object_clones":0,"num_object_copies":15400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10407,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1683},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12575","reported_seq":12616,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:05.537609+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:05.537609+0000","last_peered":"2024-09-11T01:20:05.537609+0000","last_clean":"2024-09-11T01:20:05.537609+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:05.537609+0000","last_undegraded":"2024-09-11T01:20:05.537609+0000","last_fullsized":"2024-09-11T01:20:05.537609+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31895552,"num_objects":7787,"num_object_clones":0,"num_object_copies":15574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12575,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12555","reported_seq":12672,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.021744+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:07.021744+0000","last_peered":"2024-09-11T01:20:07.021744+0000","last_clean":"2024-09-11T01:20:07.021744+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:07.021744+0000","last_undegraded":"2024-09-11T01:20:07.021744+0000","last_fullsized":"2024-09-11T01:20:07.021744+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629795,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9353,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12469","reported_seq":12679,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.037768+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:07.037768+0000","last_peered":"2024-09-11T01:20:07.037768+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:07.037768+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31695331,"num_objects":7739,"num_object_clones":0,"num_object_copies":15478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8957,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":1751}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12883","reported_seq":13106,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.086836+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:07.086836+0000","last_peered":"2024-09-11T01:20:07.086836+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:07.086836+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32478150,"num_objects":7931,"num_object_clones":0,"num_object_copies":15862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7931,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9636,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3695},{"shards":"0,2","objects":1809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12622","reported_seq":12737,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.072185+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:07.072185+0000","last_peered":"2024-09-11T01:20:07.072185+0000","last_clean":"2024-09-11T01:20:07.072185+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:07.072185+0000","last_undegraded":"2024-09-11T01:20:07.072185+0000","last_fullsized":"2024-09-11T01:20:07.072185+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31956992,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9575,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12458","reported_seq":20797,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:04.618289+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:04.618289+0000","last_peered":"2024-09-11T01:20:04.618289+0000","last_clean":"2024-09-11T01:20:04.618289+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:04.618289+0000","last_undegraded":"2024-09-11T01:20:04.618289+0000","last_fullsized":"2024-09-11T01:20:04.618289+0000","mapping_epoch":40,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31912419,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10325,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12705","reported_seq":12900,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.065539+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:07.065539+0000","last_peered":"2024-09-11T01:20:07.065539+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:07.065539+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3819},{"shards":"0,2","objects":1701}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12411","reported_seq":12551,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.504402+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:05.504402+0000","last_peered":"2024-09-11T01:20:05.504402+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:05.504402+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654371,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10180,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1688},{"shards":"2","objects":3727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12677","reported_seq":12810,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.556436+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:05.556436+0000","last_peered":"2024-09-11T01:20:05.556436+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:05.556436+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32296960,"num_objects":7885,"num_object_clones":0,"num_object_copies":15770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10314,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1784},{"shards":"2","objects":3730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12576","reported_seq":18627,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:07.107512+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:07.107512+0000","last_peered":"2024-09-11T01:20:07.107512+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:07.107512+0000","last_undegraded":"2024-09-11T01:20:07.107512+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":7776,"num_object_clones":0,"num_object_copies":15552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":7776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9266,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2962,"num_bytes_recovered":12128256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6573},{"shards":"0,2","objects":1203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12741","reported_seq":12951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.112702+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:07.112702+0000","last_peered":"2024-09-11T01:20:07.112702+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:07.112702+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9134,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3753},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12767","reported_seq":12996,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.099698+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:07.099698+0000","last_peered":"2024-09-11T01:20:07.099698+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:07.099698+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32083968,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9302,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12497","reported_seq":12680,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.040883+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:07.040883+0000","last_peered":"2024-09-11T01:20:07.040883+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:07.040883+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31298502,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9154,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3515},{"shards":"0,2","objects":1732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510808822,"num_objects":124719,"num_object_clones":0,"num_object_copies":249438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40979,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":124719,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":160522,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7452,"num_bytes_recovered":29020643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1149054976,"data_stored":1148898699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676268,"kb_used_data":383580,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93695572,"statfs":{"total":96636764160,"available":95944265728,"internally_reserved":0,"allocated":392785920,"data_stored":392514543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,0,0,0,0,0,0,0,0,0,2],"upper_bound":32768},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790010,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602900,"kb_used_data":300484,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93768940,"statfs":{"total":96636764160,"available":96019394560,"internally_reserved":0,"allocated":307695616,"data_stored":307380419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800544,"kb_used_data":439876,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93571296,"statfs":{"total":96636764160,"available":95817007104,"internally_reserved":0,"allocated":450433024,"data_stored":450161642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,4,4,2,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450019328,"data_stored":449983198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307326976,"data_stored":307247031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391708672,"data_stored":391668470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12515", 'reported_seq': 12688, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.086374+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:07.086374+0000', 'last_peered': '2024-09-11T01:20:07.086374+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:07.086374+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2500", 'ondisk_log_start': "32'2500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31633408, 'num_objects': 7723, 'num_object_clones': 0, 'num_object_copies': 15446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4071, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12515, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3652}, {'shards': '0,2', 'objects': 1727}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12346", 'reported_seq': 12478, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:05.414048+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:05.414048+0000', 'last_peered': '2024-09-11T01:20:05.414048+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:05.414048+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2300", 'ondisk_log_start': "32'2300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31535128, 'num_objects': 7700, 'num_object_clones': 0, 'num_object_copies': 15400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3980, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7700, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10407, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 27, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1683}, {'shards': '2', 'objects': 3721}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12469", 'reported_seq': 12679, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.037768+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:07.037768+0000', 'last_peered': '2024-09-11T01:20:07.037768+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:07.037768+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31695331, 'num_objects': 7739, 'num_object_clones': 0, 'num_object_copies': 15478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4076, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7739, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8957, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3664}, {'shards': '0,2', 'objects': 1751}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'12883", 'reported_seq': 13106, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.086836+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:07.086836+0000', 'last_peered': '2024-09-11T01:20:07.086836+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:07.086836+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32478150, 'num_objects': 7931, 'num_object_clones': 0, 'num_object_copies': 15862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4236, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7931, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9636, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3695}, {'shards': '0,2', 'objects': 1809}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:09.553 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12705", 'reported_seq': 12900, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.065539+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:07.065539+0000', 'last_peered': '2024-09-11T01:20:07.065539+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:07.065539+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32313344, 'num_objects': 7889, 'num_object_clones': 0, 'num_object_copies': 15778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9069, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 35, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3819}, {'shards': '0,2', 'objects': 1701}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12411", 'reported_seq': 12551, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:05.504402+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:05.504402+0000', 'last_peered': '2024-09-11T01:20:05.504402+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:05.504402+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31654371, 'num_objects': 7729, 'num_object_clones': 0, 'num_object_copies': 15458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7729, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10180, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1688}, {'shards': '2', 'objects': 3727}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12677", 'reported_seq': 12810, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:05.556436+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:05.556436+0000', 'last_peered': '2024-09-11T01:20:05.556436+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:05.556436+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32296960, 'num_objects': 7885, 'num_object_clones': 0, 'num_object_copies': 15770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10314, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 22, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1784}, {'shards': '2', 'objects': 3730}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'12576", 'reported_seq': 18627, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:07.107512+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:07.107512+0000', 'last_peered': '2024-09-11T01:20:07.107512+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:07.107512+0000', 'last_undegraded': '2024-09-11T01:20:07.107512+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'2500", 'ondisk_log_start': "32'2500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31850496, 'num_objects': 7776, 'num_object_clones': 0, 'num_object_copies': 15552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1203, 'num_objects_unfound': 0, 'num_objects_dirty': 7776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9266, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2962, 'num_bytes_recovered': 12128256, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6573}, {'shards': '0,2', 'objects': 1203}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'12741", 'reported_seq': 12951, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.112702+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:07.112702+0000', 'last_peered': '2024-09-11T01:20:07.112702+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:07.112702+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32378880, 'num_objects': 7905, 'num_object_clones': 0, 'num_object_copies': 15810, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7905, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9134, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 40, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3753}, {'shards': '0,2', 'objects': 1778}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'12767", 'reported_seq': 12996, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.099698+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:07.099698+0000', 'last_peered': '2024-09-11T01:20:07.099698+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:07.099698+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32083968, 'num_objects': 7833, 'num_object_clones': 0, 'num_object_copies': 15666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4099, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7833, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9302, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3736}, {'shards': '0,2', 'objects': 1683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:09.554 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12497", 'reported_seq': 12680, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.040883+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:07.040883+0000', 'last_peered': '2024-09-11T01:20:07.040883+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:07.040883+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31298502, 'num_objects': 7643, 'num_object_clones': 0, 'num_object_copies': 15286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4130, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7643, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9154, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3515}, {'shards': '0,2', 'objects': 1732}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:09.555 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:10.245 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:10.245 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40979/249442 objects degraded (16.428%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":124721,"data_bytes":510808841,"bytes_used":2129625088,"bytes_avail":287780667392,"bytes_total":289910292480,"degraded_objects":40979,"degraded_total":249442,"degraded_ratio":0.16428267893939272,"misplaced_objects":1203,"misplaced_total":249442,"misplaced_ratio":0.004822764410163485,"recovering_objects_per_sec":42,"recovering_bytes_per_sec":150798,"recovering_keys_per_sec":0,"num_objects_recovered":515,"num_bytes_recovered":1810432,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":467},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:10.268 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:10.268 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:10.873 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:10.874 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:10.908 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-11T01:20:10.539397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":510284553,"num_objects":124593,"num_object_clones":0,"num_object_copies":249186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40979,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":124593,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":160652,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7453,"num_bytes_recovered":29020662,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160890,"ondisk_log_size":160890,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105468,"kb_used_data":1123328,"kb_used_omap":12,"kb_used_meta":982067,"kb_avail":281010052,"statfs":{"total":289910292480,"available":287754293248,"internally_reserved":0,"allocated":1150287872,"data_stored":1149429916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1005636615},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,2,1,12,6,5,0,0,0,2,0,0,0,2],"upper_bound":32768},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18071552,"num_objects":-4412,"num_object_clones":0,"num_object_copies":-8824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-40,"num_objects_misplaced":-287,"num_objects_unfound":0,"num_objects_dirty":-4412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4412,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004764"},"pg_stats":[{"pgid":"2.f","version":"41'12515","reported_seq":12688,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.086374+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:07.086374+0000","last_peered":"2024-09-11T01:20:07.086374+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:07.086374+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12515,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3652},{"shards":"0,2","objects":1727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12662","reported_seq":12738,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:05.557545+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:05.557545+0000","last_peered":"2024-09-11T01:20:05.557545+0000","last_clean":"2024-09-11T01:20:05.557545+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:05.557545+0000","last_undegraded":"2024-09-11T01:20:05.557545+0000","last_fullsized":"2024-09-11T01:20:05.557545+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32195526,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10760,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12346","reported_seq":12478,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.414048+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:05.414048+0000","last_peered":"2024-09-11T01:20:05.414048+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:05.414048+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2300","ondisk_log_start":"32'2300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31535128,"num_objects":7700,"num_object_clones":0,"num_object_copies":15400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3980,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10407,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1683},{"shards":"2","objects":3721}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12575","reported_seq":12616,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:05.537609+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:05.537609+0000","last_peered":"2024-09-11T01:20:05.537609+0000","last_clean":"2024-09-11T01:20:05.537609+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:05.537609+0000","last_undegraded":"2024-09-11T01:20:05.537609+0000","last_fullsized":"2024-09-11T01:20:05.537609+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31895552,"num_objects":7787,"num_object_clones":0,"num_object_copies":15574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12575,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12555","reported_seq":12672,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.021744+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:07.021744+0000","last_peered":"2024-09-11T01:20:07.021744+0000","last_clean":"2024-09-11T01:20:07.021744+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:07.021744+0000","last_undegraded":"2024-09-11T01:20:07.021744+0000","last_fullsized":"2024-09-11T01:20:07.021744+0000","mapping_epoch":31,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629795,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9353,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12469","reported_seq":12679,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.037768+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:07.037768+0000","last_peered":"2024-09-11T01:20:07.037768+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:07.037768+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31695331,"num_objects":7739,"num_object_clones":0,"num_object_copies":15478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8957,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3664},{"shards":"0,2","objects":1751}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'12883","reported_seq":13106,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.086836+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:07.086836+0000","last_peered":"2024-09-11T01:20:07.086836+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:07.086836+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32478150,"num_objects":7931,"num_object_clones":0,"num_object_copies":15862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7931,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9636,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3695},{"shards":"0,2","objects":1809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12622","reported_seq":12737,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.072185+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:07.072185+0000","last_peered":"2024-09-11T01:20:07.072185+0000","last_clean":"2024-09-11T01:20:07.072185+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:07.072185+0000","last_undegraded":"2024-09-11T01:20:07.072185+0000","last_fullsized":"2024-09-11T01:20:07.072185+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31956992,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9575,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12586","reported_seq":20925,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.601103+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:09.601103+0000","last_peered":"2024-09-11T01:20:09.601103+0000","last_clean":"2024-09-11T01:20:09.601103+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:09.601103+0000","last_undegraded":"2024-09-11T01:20:09.601103+0000","last_fullsized":"2024-09-11T01:20:09.601103+0000","mapping_epoch":40,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10453,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12705","reported_seq":12900,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.065539+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:07.065539+0000","last_peered":"2024-09-11T01:20:07.065539+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:07.065539+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32313344,"num_objects":7889,"num_object_clones":0,"num_object_copies":15778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3819},{"shards":"0,2","objects":1701}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12411","reported_seq":12551,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.504402+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:05.504402+0000","last_peered":"2024-09-11T01:20:05.504402+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:05.504402+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654371,"num_objects":7729,"num_object_clones":0,"num_object_copies":15458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10180,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1688},{"shards":"2","objects":3727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12677","reported_seq":12810,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:05.556436+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:05.556436+0000","last_peered":"2024-09-11T01:20:05.556436+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:05.556436+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32296960,"num_objects":7885,"num_object_clones":0,"num_object_copies":15770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10314,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1784},{"shards":"2","objects":3730}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12576","reported_seq":18627,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:07.107512+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:07.107512+0000","last_peered":"2024-09-11T01:20:07.107512+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:07.107512+0000","last_undegraded":"2024-09-11T01:20:07.107512+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31850496,"num_objects":7776,"num_object_clones":0,"num_object_copies":15552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":7776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9266,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2962,"num_bytes_recovered":12128256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6573},{"shards":"0,2","objects":1203}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12741","reported_seq":12951,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.112702+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:07.112702+0000","last_peered":"2024-09-11T01:20:07.112702+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:07.112702+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32378880,"num_objects":7905,"num_object_clones":0,"num_object_copies":15810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9134,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3753},{"shards":"0,2","objects":1778}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12767","reported_seq":12996,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.099698+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:07.099698+0000","last_peered":"2024-09-11T01:20:07.099698+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:07.099698+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32083968,"num_objects":7833,"num_object_clones":0,"num_object_copies":15666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4099,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9302,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":1683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12497","reported_seq":12680,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.040883+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:07.040883+0000","last_peered":"2024-09-11T01:20:07.040883+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:07.040883+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31298502,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4130,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9154,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3515},{"shards":"0,2","objects":1732}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":510284534,"num_objects":124591,"num_object_clones":0,"num_object_copies":249182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40979,"num_objects_misplaced":1203,"num_objects_unfound":0,"num_objects_dirty":124591,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":160650,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7452,"num_bytes_recovered":29020643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1148432384,"data_stored":1148276107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676268,"kb_used_data":383580,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93695572,"statfs":{"total":96636764160,"available":95944265728,"internally_reserved":0,"allocated":392785920,"data_stored":392514543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,0,0,0,0,0,0,0,0,0,2],"upper_bound":32768},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790011,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628656,"kb_used_data":299872,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93743184,"statfs":{"total":96636764160,"available":95993020416,"internally_reserved":0,"allocated":307068928,"data_stored":306753731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800544,"kb_used_data":439876,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93571296,"statfs":{"total":96636764160,"available":95817007104,"internally_reserved":0,"allocated":450433024,"data_stored":450161642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,2,0,4,4,2,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450019328,"data_stored":449983198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306704384,"data_stored":306624439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391708672,"data_stored":391668470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:13.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:14.476 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:14.478 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:14.502 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":241,"stamp":"2024-09-11T01:20:12.540322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502010633,"num_objects":122573,"num_object_clones":0,"num_object_copies":245146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40951,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":122573,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":162672,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7600,"num_bytes_recovered":29508086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2086428,"kb_used_data":1112608,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":281029092,"statfs":{"total":289910292480,"available":287773790208,"internally_reserved":0,"allocated":1139310592,"data_stored":1138452636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":997116935},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,11,10,6,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25858048,"num_objects":-6313,"num_object_clones":0,"num_object_copies":-12626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-68,"num_objects_misplaced":-406,"num_objects_unfound":0,"num_objects_dirty":-6313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6313,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":474,"num_bytes_recovered":1662976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004958"},"pg_stats":[{"pgid":"2.f","version":"41'12672","reported_seq":12851,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.175682+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:12.175682+0000","last_peered":"2024-09-11T01:20:12.175682+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:12.175682+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30990336,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3498},{"shards":"0,2","objects":1570}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12782","reported_seq":12858,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:10.503797+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:10.503797+0000","last_peered":"2024-09-11T01:20:10.503797+0000","last_clean":"2024-09-11T01:20:10.503797+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:10.503797+0000","last_undegraded":"2024-09-11T01:20:10.503797+0000","last_fullsized":"2024-09-11T01:20:10.503797+0000","mapping_epoch":31,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31704006,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10880,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12491","reported_seq":12625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.561085+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:10.561085+0000","last_peered":"2024-09-11T01:20:10.561085+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:10.561085+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30941208,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10552,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1538},{"shards":"2","objects":3577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12689","reported_seq":12730,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:10.461249+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:10.461249+0000","last_peered":"2024-09-11T01:20:10.461249+0000","last_clean":"2024-09-11T01:20:10.461249+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:10.461249+0000","last_undegraded":"2024-09-11T01:20:10.461249+0000","last_fullsized":"2024-09-11T01:20:10.461249+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12689,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12683","reported_seq":12800,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.173121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:12.173121+0000","last_peered":"2024-09-11T01:20:12.173121+0000","last_clean":"2024-09-11T01:20:12.173121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:12.173121+0000","last_undegraded":"2024-09-11T01:20:12.173121+0000","last_fullsized":"2024-09-11T01:20:12.173121+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31105507,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9481,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12605","reported_seq":12823,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.095319+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:12.095319+0000","last_peered":"2024-09-11T01:20:12.095319+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:12.095319+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31138275,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9093,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13018","reported_seq":13245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.144345+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:12.144345+0000","last_peered":"2024-09-11T01:20:12.144345+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:12.144345+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31925190,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9771,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3562},{"shards":"0,2","objects":1674}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12781","reported_seq":12896,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.127306+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:12.127306+0000","last_peered":"2024-09-11T01:20:12.127306+0000","last_clean":"2024-09-11T01:20:12.127306+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:12.127306+0000","last_undegraded":"2024-09-11T01:20:12.127306+0000","last_fullsized":"2024-09-11T01:20:12.127306+0000","mapping_epoch":31,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31305728,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9734,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12586","reported_seq":20925,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.601103+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:09.601103+0000","last_peered":"2024-09-11T01:20:09.601103+0000","last_clean":"2024-09-11T01:20:09.601103+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:09.601103+0000","last_undegraded":"2024-09-11T01:20:09.601103+0000","last_fullsized":"2024-09-11T01:20:09.601103+0000","mapping_epoch":40,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10453,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12824","reported_seq":13035,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.147860+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:12.147860+0000","last_peered":"2024-09-11T01:20:12.147860+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:12.147860+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3708},{"shards":"0,2","objects":1582}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12525","reported_seq":12665,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.505082+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:10.505082+0000","last_peered":"2024-09-11T01:20:10.505082+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:10.505082+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10294,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1574},{"shards":"2","objects":3613}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12813","reported_seq":12950,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.536524+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:10.536524+0000","last_peered":"2024-09-11T01:20:10.536524+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:10.536524+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1648},{"shards":"2","objects":3596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12729","reported_seq":19018,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:12.177489+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:12.177489+0000","last_peered":"2024-09-11T01:20:12.177489+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:12.177489+0000","last_undegraded":"2024-09-11T01:20:12.177489+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31223808,"num_objects":7623,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":7623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9419,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3081,"num_bytes_recovered":12615680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6539},{"shards":"0,2","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12884","reported_seq":13096,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.146994+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:12.146994+0000","last_peered":"2024-09-11T01:20:12.146994+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:12.146994+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793152,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3611},{"shards":"0,2","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12907","reported_seq":13148,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.086731+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:12.086731+0000","last_peered":"2024-09-11T01:20:12.086731+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:12.086731+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9442,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3601},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12618","reported_seq":12805,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.086703+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:12.086703+0000","last_peered":"2024-09-11T01:20:12.086703+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:12.086703+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30802886,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9275,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3396},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502010614,"num_objects":122571,"num_object_clones":0,"num_object_copies":245142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40951,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":122571,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":162670,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7599,"num_bytes_recovered":29508067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1137291264,"data_stored":1137134987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659588,"kb_used_data":379380,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93712252,"statfs":{"total":96636764160,"available":95961346048,"internally_reserved":0,"allocated":388485120,"data_stored":388213743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790011,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628656,"kb_used_data":299872,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93743184,"statfs":{"total":96636764160,"available":95993020416,"internally_reserved":0,"allocated":307068928,"data_stored":306753731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798184,"kb_used_data":433356,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93573656,"statfs":{"total":96636764160,"available":95819423744,"internally_reserved":0,"allocated":443756544,"data_stored":443485162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,3,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443215872,"data_stored":443179742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306704384,"data_stored":306624439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387371008,"data_stored":387330806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:14.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:15.081 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:15.082 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:15.103 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-11T01:20:14.541104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502010633,"num_objects":122573,"num_object_clones":0,"num_object_copies":245146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40951,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":122573,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":162672,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7600,"num_bytes_recovered":29508086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2086428,"kb_used_data":1112608,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":281029092,"statfs":{"total":289910292480,"available":287773790208,"internally_reserved":0,"allocated":1139310592,"data_stored":1138452636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":997116935},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,11,10,6,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17633280,"num_objects":-4305,"num_object_clones":0,"num_object_copies":-8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-62,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005079"},"pg_stats":[{"pgid":"2.f","version":"41'12672","reported_seq":12851,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.175682+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:12.175682+0000","last_peered":"2024-09-11T01:20:12.175682+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:12.175682+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30990336,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3498},{"shards":"0,2","objects":1570}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12782","reported_seq":12858,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:10.503797+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:10.503797+0000","last_peered":"2024-09-11T01:20:10.503797+0000","last_clean":"2024-09-11T01:20:10.503797+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:10.503797+0000","last_undegraded":"2024-09-11T01:20:10.503797+0000","last_fullsized":"2024-09-11T01:20:10.503797+0000","mapping_epoch":31,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31704006,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10880,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12491","reported_seq":12625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.561085+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:10.561085+0000","last_peered":"2024-09-11T01:20:10.561085+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:10.561085+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30941208,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10552,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1538},{"shards":"2","objects":3577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12689","reported_seq":12730,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:10.461249+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:10.461249+0000","last_peered":"2024-09-11T01:20:10.461249+0000","last_clean":"2024-09-11T01:20:10.461249+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:10.461249+0000","last_undegraded":"2024-09-11T01:20:10.461249+0000","last_fullsized":"2024-09-11T01:20:10.461249+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12689,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12683","reported_seq":12800,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.173121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:12.173121+0000","last_peered":"2024-09-11T01:20:12.173121+0000","last_clean":"2024-09-11T01:20:12.173121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:12.173121+0000","last_undegraded":"2024-09-11T01:20:12.173121+0000","last_fullsized":"2024-09-11T01:20:12.173121+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31105507,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9481,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12605","reported_seq":12823,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.095319+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:12.095319+0000","last_peered":"2024-09-11T01:20:12.095319+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:12.095319+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31138275,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9093,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13018","reported_seq":13245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.144345+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:12.144345+0000","last_peered":"2024-09-11T01:20:12.144345+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:12.144345+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31925190,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9771,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3562},{"shards":"0,2","objects":1674}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12781","reported_seq":12896,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.127306+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:12.127306+0000","last_peered":"2024-09-11T01:20:12.127306+0000","last_clean":"2024-09-11T01:20:12.127306+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:12.127306+0000","last_undegraded":"2024-09-11T01:20:12.127306+0000","last_fullsized":"2024-09-11T01:20:12.127306+0000","mapping_epoch":31,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31305728,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9734,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12586","reported_seq":20925,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.601103+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:09.601103+0000","last_peered":"2024-09-11T01:20:09.601103+0000","last_clean":"2024-09-11T01:20:09.601103+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:09.601103+0000","last_undegraded":"2024-09-11T01:20:09.601103+0000","last_fullsized":"2024-09-11T01:20:09.601103+0000","mapping_epoch":40,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10453,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12824","reported_seq":13035,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.147860+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:12.147860+0000","last_peered":"2024-09-11T01:20:12.147860+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:12.147860+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3708},{"shards":"0,2","objects":1582}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12525","reported_seq":12665,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.505082+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:10.505082+0000","last_peered":"2024-09-11T01:20:10.505082+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:10.505082+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10294,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1574},{"shards":"2","objects":3613}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12813","reported_seq":12950,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.536524+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:10.536524+0000","last_peered":"2024-09-11T01:20:10.536524+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:10.536524+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1648},{"shards":"2","objects":3596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12729","reported_seq":19018,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:12.177489+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:12.177489+0000","last_peered":"2024-09-11T01:20:12.177489+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:12.177489+0000","last_undegraded":"2024-09-11T01:20:12.177489+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31223808,"num_objects":7623,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":7623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9419,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3081,"num_bytes_recovered":12615680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6539},{"shards":"0,2","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12884","reported_seq":13096,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.146994+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:12.146994+0000","last_peered":"2024-09-11T01:20:12.146994+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:12.146994+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793152,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3611},{"shards":"0,2","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12907","reported_seq":13148,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.086731+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:12.086731+0000","last_peered":"2024-09-11T01:20:12.086731+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:12.086731+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9442,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3601},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12618","reported_seq":12805,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.086703+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:12.086703+0000","last_peered":"2024-09-11T01:20:12.086703+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:12.086703+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30802886,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9275,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3396},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502010614,"num_objects":122571,"num_object_clones":0,"num_object_copies":245142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40951,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":122571,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":162670,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7599,"num_bytes_recovered":29508067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1137291264,"data_stored":1137134987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659588,"kb_used_data":379380,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93712252,"statfs":{"total":96636764160,"available":95961346048,"internally_reserved":0,"allocated":388485120,"data_stored":388213743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790011,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628656,"kb_used_data":299872,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93743184,"statfs":{"total":96636764160,"available":95993020416,"internally_reserved":0,"allocated":307068928,"data_stored":306753731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798184,"kb_used_data":433356,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93573656,"statfs":{"total":96636764160,"available":95819423744,"internally_reserved":0,"allocated":443756544,"data_stored":443485162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,3,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443215872,"data_stored":443179742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306704384,"data_stored":306624439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387371008,"data_stored":387330806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:15.104 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:15.104 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12672", 'reported_seq': 12851, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.175682+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:12.175682+0000', 'last_peered': '2024-09-11T01:20:12.175682+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:12.175682+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30990336, 'num_objects': 7566, 'num_object_clones': 0, 'num_object_copies': 15132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4068, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12672, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3498}, {'shards': '0,2', 'objects': 1570}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.104 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:15.104 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12491", 'reported_seq': 12625, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:10.561085+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:10.561085+0000', 'last_peered': '2024-09-11T01:20:10.561085+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:10.561085+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2400", 'ondisk_log_start': "32'2400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30941208, 'num_objects': 7555, 'num_object_clones': 0, 'num_object_copies': 15110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7555, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10552, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1538}, {'shards': '2', 'objects': 3577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:15.104 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12605", 'reported_seq': 12823, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.095319+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:12.095319+0000', 'last_peered': '2024-09-11T01:20:12.095319+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:12.095319+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31138275, 'num_objects': 7603, 'num_object_clones': 0, 'num_object_copies': 15206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7603, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9093, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 44, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3532}, {'shards': '0,2', 'objects': 1615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13018", 'reported_seq': 13245, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.144345+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:12.144345+0000', 'last_peered': '2024-09-11T01:20:12.144345+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:12.144345+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31925190, 'num_objects': 7796, 'num_object_clones': 0, 'num_object_copies': 15592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4234, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7796, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9771, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3562}, {'shards': '0,2', 'objects': 1674}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12824", 'reported_seq': 13035, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.147860+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:12.147860+0000', 'last_peered': '2024-09-11T01:20:12.147860+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:12.147860+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31825920, 'num_objects': 7770, 'num_object_clones': 0, 'num_object_copies': 15540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4064, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9188, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3708}, {'shards': '0,2', 'objects': 1582}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12525", 'reported_seq': 12665, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:10.505082+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:10.505082+0000', 'last_peered': '2024-09-11T01:20:10.505082+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:10.505082+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2500", 'ondisk_log_start': "32'2500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31187427, 'num_objects': 7615, 'num_object_clones': 0, 'num_object_copies': 15230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7615, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10294, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1574}, {'shards': '2', 'objects': 3613}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12813", 'reported_seq': 12950, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:10.536524+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:10.536524+0000', 'last_peered': '2024-09-11T01:20:10.536524+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:10.536524+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31739904, 'num_objects': 7749, 'num_object_clones': 0, 'num_object_copies': 15498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10450, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1648}, {'shards': '2', 'objects': 3596}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:15.105 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'12729", 'reported_seq': 19018, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:12.177489+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:12.177489+0000', 'last_peered': '2024-09-11T01:20:12.177489+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:12.177489+0000', 'last_undegraded': '2024-09-11T01:20:12.177489+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31223808, 'num_objects': 7623, 'num_object_clones': 0, 'num_object_copies': 15246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1084, 'num_objects_unfound': 0, 'num_objects_dirty': 7623, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9419, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3081, 'num_bytes_recovered': 12615680, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6539}, {'shards': '0,2', 'objects': 1084}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.106 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:15.106 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'12884", 'reported_seq': 13096, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.146994+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:12.146994+0000', 'last_peered': '2024-09-11T01:20:12.146994+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:12.146994+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31793152, 'num_objects': 7762, 'num_object_clones': 0, 'num_object_copies': 15524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9277, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3611}, {'shards': '0,2', 'objects': 1635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.106 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:15.106 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'12907", 'reported_seq': 13148, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.086731+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:12.086731+0000', 'last_peered': '2024-09-11T01:20:12.086731+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:12.086731+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'2900", 'ondisk_log_start': "32'2900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31510528, 'num_objects': 7693, 'num_object_clones': 0, 'num_object_copies': 15386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4094, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7693, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9442, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3601}, {'shards': '0,2', 'objects': 1543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.106 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:15.106 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12618", 'reported_seq': 12805, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.086703+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:12.086703+0000', 'last_peered': '2024-09-11T01:20:12.086703+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:12.086703+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30802886, 'num_objects': 7522, 'num_object_clones': 0, 'num_object_copies': 15044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4128, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7522, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9275, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3396}, {'shards': '0,2', 'objects': 1611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:15.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:15.818 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:15.818 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40951/245146 objects degraded (16.705%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":416,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122573,"data_bytes":502010633,"bytes_used":2136502272,"bytes_avail":287773790208,"bytes_total":289910292480,"degraded_objects":40951,"degraded_total":245146,"degraded_ratio":0.16704739216630091,"misplaced_objects":1084,"misplaced_total":245146,"misplaced_ratio":0.0044218547314661466,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":90756,"recovering_keys_per_sec":0,"num_objects_recovered":328,"num_bytes_recovered":1089536,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":358},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:15.841 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:15.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:16.404 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:16.404 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:16.431 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-11T01:20:14.541104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502010633,"num_objects":122573,"num_object_clones":0,"num_object_copies":245146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40951,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":122573,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":162672,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7600,"num_bytes_recovered":29508086,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160810,"ondisk_log_size":160810,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2086428,"kb_used_data":1112608,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":281029092,"statfs":{"total":289910292480,"available":287773790208,"internally_reserved":0,"allocated":1139310592,"data_stored":1138452636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":997116935},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,11,10,6,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17633280,"num_objects":-4305,"num_object_clones":0,"num_object_copies":-8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-62,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4305,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005079"},"pg_stats":[{"pgid":"2.f","version":"41'12672","reported_seq":12851,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.175682+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:12.175682+0000","last_peered":"2024-09-11T01:20:12.175682+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:12.175682+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30990336,"num_objects":7566,"num_object_clones":0,"num_object_copies":15132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3498},{"shards":"0,2","objects":1570}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12782","reported_seq":12858,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:10.503797+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:10.503797+0000","last_peered":"2024-09-11T01:20:10.503797+0000","last_clean":"2024-09-11T01:20:10.503797+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:10.503797+0000","last_undegraded":"2024-09-11T01:20:10.503797+0000","last_fullsized":"2024-09-11T01:20:10.503797+0000","mapping_epoch":31,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31704006,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10880,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12491","reported_seq":12625,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.561085+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:10.561085+0000","last_peered":"2024-09-11T01:20:10.561085+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:10.561085+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2400","ondisk_log_start":"32'2400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30941208,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10552,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1538},{"shards":"2","objects":3577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12689","reported_seq":12730,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:10.461249+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:10.461249+0000","last_peered":"2024-09-11T01:20:10.461249+0000","last_clean":"2024-09-11T01:20:10.461249+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:10.461249+0000","last_undegraded":"2024-09-11T01:20:10.461249+0000","last_fullsized":"2024-09-11T01:20:10.461249+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31428608,"num_objects":7673,"num_object_clones":0,"num_object_copies":15346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12689,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12683","reported_seq":12800,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.173121+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:12.173121+0000","last_peered":"2024-09-11T01:20:12.173121+0000","last_clean":"2024-09-11T01:20:12.173121+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:12.173121+0000","last_undegraded":"2024-09-11T01:20:12.173121+0000","last_fullsized":"2024-09-11T01:20:12.173121+0000","mapping_epoch":31,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31105507,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9481,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12605","reported_seq":12823,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.095319+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:12.095319+0000","last_peered":"2024-09-11T01:20:12.095319+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:12.095319+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31138275,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9093,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3532},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13018","reported_seq":13245,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.144345+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:12.144345+0000","last_peered":"2024-09-11T01:20:12.144345+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:12.144345+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31925190,"num_objects":7796,"num_object_clones":0,"num_object_copies":15592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7796,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9771,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3562},{"shards":"0,2","objects":1674}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12781","reported_seq":12896,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.127306+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:12.127306+0000","last_peered":"2024-09-11T01:20:12.127306+0000","last_clean":"2024-09-11T01:20:12.127306+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:12.127306+0000","last_undegraded":"2024-09-11T01:20:12.127306+0000","last_fullsized":"2024-09-11T01:20:12.127306+0000","mapping_epoch":31,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31305728,"num_objects":7643,"num_object_clones":0,"num_object_copies":15286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9734,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12586","reported_seq":20925,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.601103+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:09.601103+0000","last_peered":"2024-09-11T01:20:09.601103+0000","last_clean":"2024-09-11T01:20:09.601103+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:09.601103+0000","last_undegraded":"2024-09-11T01:20:09.601103+0000","last_fullsized":"2024-09-11T01:20:09.601103+0000","mapping_epoch":40,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31388131,"num_objects":7664,"num_object_clones":0,"num_object_copies":15328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10453,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12824","reported_seq":13035,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.147860+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:12.147860+0000","last_peered":"2024-09-11T01:20:12.147860+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:12.147860+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9188,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3708},{"shards":"0,2","objects":1582}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12525","reported_seq":12665,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.505082+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:10.505082+0000","last_peered":"2024-09-11T01:20:10.505082+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:10.505082+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2500","ondisk_log_start":"32'2500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31187427,"num_objects":7615,"num_object_clones":0,"num_object_copies":15230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10294,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1574},{"shards":"2","objects":3613}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12813","reported_seq":12950,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:10.536524+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:10.536524+0000","last_peered":"2024-09-11T01:20:10.536524+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:10.536524+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31739904,"num_objects":7749,"num_object_clones":0,"num_object_copies":15498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1648},{"shards":"2","objects":3596}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12729","reported_seq":19018,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:12.177489+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:12.177489+0000","last_peered":"2024-09-11T01:20:12.177489+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:12.177489+0000","last_undegraded":"2024-09-11T01:20:12.177489+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31223808,"num_objects":7623,"num_object_clones":0,"num_object_copies":15246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":7623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9419,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3081,"num_bytes_recovered":12615680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6539},{"shards":"0,2","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'12884","reported_seq":13096,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.146994+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:12.146994+0000","last_peered":"2024-09-11T01:20:12.146994+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:12.146994+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793152,"num_objects":7762,"num_object_clones":0,"num_object_copies":15524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9277,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3611},{"shards":"0,2","objects":1635}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'12907","reported_seq":13148,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.086731+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:12.086731+0000","last_peered":"2024-09-11T01:20:12.086731+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:12.086731+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31510528,"num_objects":7693,"num_object_clones":0,"num_object_copies":15386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4094,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9442,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3601},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12618","reported_seq":12805,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.086703+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:12.086703+0000","last_peered":"2024-09-11T01:20:12.086703+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:12.086703+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30802886,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4128,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9275,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3396},{"shards":"0,2","objects":1611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502010614,"num_objects":122571,"num_object_clones":0,"num_object_copies":245142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40951,"num_objects_misplaced":1084,"num_objects_unfound":0,"num_objects_dirty":122571,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":162670,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7599,"num_bytes_recovered":29508067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1137291264,"data_stored":1137134987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659588,"kb_used_data":379380,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93712252,"statfs":{"total":96636764160,"available":95961346048,"internally_reserved":0,"allocated":388485120,"data_stored":388213743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790011,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628656,"kb_used_data":299872,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93743184,"statfs":{"total":96636764160,"available":95993020416,"internally_reserved":0,"allocated":307068928,"data_stored":306753731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798184,"kb_used_data":433356,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93573656,"statfs":{"total":96636764160,"available":95819423744,"internally_reserved":0,"allocated":443756544,"data_stored":443485162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,7,3,0,0,0,0,4],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443215872,"data_stored":443179742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306704384,"data_stored":306624439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387371008,"data_stored":387330806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:19.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:20.001 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:20.001 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:20.024 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-11T01:20:18.542639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493048585,"num_objects":120385,"num_object_clones":0,"num_object_copies":240770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40922,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":120385,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":164860,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7776,"num_bytes_recovered":30110198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160598,"ondisk_log_size":160598,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2058424,"kb_used_data":1101252,"kb_used_omap":12,"kb_used_meta":957107,"kb_avail":281057096,"statfs":{"total":289910292480,"available":287802466304,"internally_reserved":0,"allocated":1127682048,"data_stored":1126824092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":980077575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,7,8,4,1,0,0,2,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23203840,"num_objects":-5665,"num_object_clones":0,"num_object_copies":-11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-91,"num_objects_misplaced":-413,"num_objects_unfound":0,"num_objects_dirty":-5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":1691648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004882"},"pg_stats":[{"pgid":"2.f","version":"41'12807","reported_seq":12992,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.188238+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:17.188238+0000","last_peered":"2024-09-11T01:20:17.188238+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:17.188238+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30437376,"num_objects":7431,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12807,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3366},{"shards":"0,2","objects":1435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12918","reported_seq":12994,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:15.548904+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:15.548904+0000","last_peered":"2024-09-11T01:20:15.548904+0000","last_clean":"2024-09-11T01:20:15.548904+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:15.548904+0000","last_undegraded":"2024-09-11T01:20:15.548904+0000","last_fullsized":"2024-09-11T01:20:15.548904+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146950,"num_objects":7606,"num_object_clones":0,"num_object_copies":15212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7606,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11016,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12612","reported_seq":12746,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.472939+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:15.472939+0000","last_peered":"2024-09-11T01:20:15.472939+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:15.472939+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445592,"num_objects":7434,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10673,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1417},{"shards":"2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12832","reported_seq":12873,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:15.534380+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:15.534380+0000","last_peered":"2024-09-11T01:20:15.534380+0000","last_clean":"2024-09-11T01:20:15.534380+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:15.534380+0000","last_undegraded":"2024-09-11T01:20:15.534380+0000","last_fullsized":"2024-09-11T01:20:15.534380+0000","mapping_epoch":31,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30842880,"num_objects":7530,"num_object_clones":0,"num_object_copies":15060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12821","reported_seq":12938,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:17.211794+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:17.211794+0000","last_peered":"2024-09-11T01:20:17.211794+0000","last_clean":"2024-09-11T01:20:17.211794+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:17.211794+0000","last_undegraded":"2024-09-11T01:20:17.211794+0000","last_fullsized":"2024-09-11T01:20:17.211794+0000","mapping_epoch":31,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30540259,"num_objects":7457,"num_object_clones":0,"num_object_copies":14914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9619,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12743","reported_seq":12967,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.199948+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:17.199948+0000","last_peered":"2024-09-11T01:20:17.199948+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:17.199948+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573027,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9231,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3397},{"shards":"0,2","objects":1477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13178","reported_seq":13412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.200382+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:17.200382+0000","last_peered":"2024-09-11T01:20:17.200382+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:17.200382+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31269830,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9931,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3406},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12919","reported_seq":13034,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:16.990653+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:16.990653+0000","last_peered":"2024-09-11T01:20:16.990653+0000","last_clean":"2024-09-11T01:20:16.990653+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:16.990653+0000","last_undegraded":"2024-09-11T01:20:16.990653+0000","last_fullsized":"2024-09-11T01:20:16.990653+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12727","reported_seq":21066,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.659614+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:14.659614+0000","last_peered":"2024-09-11T01:20:14.659614+0000","last_clean":"2024-09-11T01:20:14.659614+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:14.659614+0000","last_undegraded":"2024-09-11T01:20:14.659614+0000","last_fullsized":"2024-09-11T01:20:14.659614+0000","mapping_epoch":40,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810595,"num_objects":7523,"num_object_clones":0,"num_object_copies":15046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7523,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10594,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12960","reported_seq":13187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.129380+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:17.129380+0000","last_peered":"2024-09-11T01:20:17.129380+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:17.129380+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9324,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3580},{"shards":"0,2","objects":1446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12647","reported_seq":12789,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.509944+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:15.509944+0000","last_peered":"2024-09-11T01:20:15.509944+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:15.509944+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687715,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10416,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1452},{"shards":"2","objects":3492}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12940","reported_seq":13076,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.581112+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:15.581112+0000","last_peered":"2024-09-11T01:20:15.581112+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:15.581112+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31219712,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10577,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1521},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12883","reported_seq":19464,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:17.208676+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:17.208676+0000","last_peered":"2024-09-11T01:20:17.208676+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:17.208676+0000","last_undegraded":"2024-09-11T01:20:17.208676+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30593024,"num_objects":7469,"num_object_clones":0,"num_object_copies":14938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":7469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9573,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3228,"num_bytes_recovered":13217792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6532},{"shards":"0,2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13027","reported_seq":13247,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.196586+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:17.196586+0000","last_peered":"2024-09-11T01:20:17.196586+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:17.196586+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31207424,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9420,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":1492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13022","reported_seq":13267,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.185930+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:17.185930+0000","last_peered":"2024-09-11T01:20:17.185930+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:17.185930+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3488},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12759","reported_seq":12954,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.167549+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:17.167549+0000","last_peered":"2024-09-11T01:20:17.167549+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:17.167549+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30225350,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9416,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3259},{"shards":"0,2","objects":1470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493048566,"num_objects":120383,"num_object_clones":0,"num_object_copies":240766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40922,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":120383,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":164858,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7775,"num_bytes_recovered":30110179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125552128,"data_stored":1125395851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160595,"ondisk_log_size":160595,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659612,"kb_used_data":375244,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93712228,"statfs":{"total":96636764160,"available":95961321472,"internally_reserved":0,"allocated":384249856,"data_stored":383978479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790012,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":615492,"kb_used_data":299188,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93756348,"statfs":{"total":96636764160,"available":96006500352,"internally_reserved":0,"allocated":306368512,"data_stored":306053315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783320,"kb_used_data":426820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93588520,"statfs":{"total":96636764160,"available":95834644480,"internally_reserved":0,"allocated":437063680,"data_stored":436792298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,3,1,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436441088,"data_stored":436404958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306028544,"data_stored":305948599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383082496,"data_stored":383042294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:20.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:20.602 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:20.605 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:20.627 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-11T01:20:20.543277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":492487433,"num_objects":120248,"num_object_clones":0,"num_object_copies":240496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40922,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":120248,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":164997,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7776,"num_bytes_recovered":30110198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160635,"ondisk_log_size":160635,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2066252,"kb_used_data":1100756,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281049268,"statfs":{"total":289910292480,"available":287794450432,"internally_reserved":0,"allocated":1127174144,"data_stored":1126316188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":988597255},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,8,6,5,1,0,0,2,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18321408,"num_objects":-4473,"num_object_clones":0,"num_object_copies":-8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004677"},"pg_stats":[{"pgid":"2.f","version":"41'12807","reported_seq":12992,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.188238+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:17.188238+0000","last_peered":"2024-09-11T01:20:17.188238+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:17.188238+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30437376,"num_objects":7431,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12807,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3366},{"shards":"0,2","objects":1435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12918","reported_seq":12994,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:15.548904+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:15.548904+0000","last_peered":"2024-09-11T01:20:15.548904+0000","last_clean":"2024-09-11T01:20:15.548904+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:15.548904+0000","last_undegraded":"2024-09-11T01:20:15.548904+0000","last_fullsized":"2024-09-11T01:20:15.548904+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146950,"num_objects":7606,"num_object_clones":0,"num_object_copies":15212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7606,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11016,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12612","reported_seq":12746,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.472939+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:15.472939+0000","last_peered":"2024-09-11T01:20:15.472939+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:15.472939+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445592,"num_objects":7434,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10673,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1417},{"shards":"2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12832","reported_seq":12873,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:15.534380+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:15.534380+0000","last_peered":"2024-09-11T01:20:15.534380+0000","last_clean":"2024-09-11T01:20:15.534380+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:15.534380+0000","last_undegraded":"2024-09-11T01:20:15.534380+0000","last_fullsized":"2024-09-11T01:20:15.534380+0000","mapping_epoch":31,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30842880,"num_objects":7530,"num_object_clones":0,"num_object_copies":15060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12821","reported_seq":12938,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:17.211794+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:17.211794+0000","last_peered":"2024-09-11T01:20:17.211794+0000","last_clean":"2024-09-11T01:20:17.211794+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:17.211794+0000","last_undegraded":"2024-09-11T01:20:17.211794+0000","last_fullsized":"2024-09-11T01:20:17.211794+0000","mapping_epoch":31,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30540259,"num_objects":7457,"num_object_clones":0,"num_object_copies":14914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9619,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12743","reported_seq":12967,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.199948+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:17.199948+0000","last_peered":"2024-09-11T01:20:17.199948+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:17.199948+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573027,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9231,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3397},{"shards":"0,2","objects":1477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13178","reported_seq":13412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.200382+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:17.200382+0000","last_peered":"2024-09-11T01:20:17.200382+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:17.200382+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31269830,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9931,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3406},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12919","reported_seq":13034,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:16.990653+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:16.990653+0000","last_peered":"2024-09-11T01:20:16.990653+0000","last_clean":"2024-09-11T01:20:16.990653+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:16.990653+0000","last_undegraded":"2024-09-11T01:20:16.990653+0000","last_fullsized":"2024-09-11T01:20:16.990653+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12864","reported_seq":21203,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.736682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:19.736682+0000","last_peered":"2024-09-11T01:20:19.736682+0000","last_clean":"2024-09-11T01:20:19.736682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:19.736682+0000","last_undegraded":"2024-09-11T01:20:19.736682+0000","last_fullsized":"2024-09-11T01:20:19.736682+0000","mapping_epoch":40,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30249443,"num_objects":7386,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10731,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12960","reported_seq":13187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.129380+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:17.129380+0000","last_peered":"2024-09-11T01:20:17.129380+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:17.129380+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9324,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3580},{"shards":"0,2","objects":1446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12647","reported_seq":12789,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.509944+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:15.509944+0000","last_peered":"2024-09-11T01:20:15.509944+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:15.509944+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687715,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10416,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1452},{"shards":"2","objects":3492}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12940","reported_seq":13076,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.581112+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:15.581112+0000","last_peered":"2024-09-11T01:20:15.581112+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:15.581112+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31219712,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10577,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1521},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12883","reported_seq":19464,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:17.208676+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:17.208676+0000","last_peered":"2024-09-11T01:20:17.208676+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:17.208676+0000","last_undegraded":"2024-09-11T01:20:17.208676+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30593024,"num_objects":7469,"num_object_clones":0,"num_object_copies":14938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":7469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9573,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3228,"num_bytes_recovered":13217792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6532},{"shards":"0,2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13027","reported_seq":13247,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.196586+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:17.196586+0000","last_peered":"2024-09-11T01:20:17.196586+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:17.196586+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31207424,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9420,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":1492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13022","reported_seq":13267,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.185930+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:17.185930+0000","last_peered":"2024-09-11T01:20:17.185930+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:17.185930+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3488},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12759","reported_seq":12954,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.167549+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:17.167549+0000","last_peered":"2024-09-11T01:20:17.167549+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:17.167549+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30225350,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9416,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3259},{"shards":"0,2","objects":1470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":492487414,"num_objects":120246,"num_object_clones":0,"num_object_copies":240492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40922,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":120246,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":164995,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7775,"num_bytes_recovered":30110179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125031936,"data_stored":1124875659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659612,"kb_used_data":375244,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93712228,"statfs":{"total":96636764160,"available":95961321472,"internally_reserved":0,"allocated":384249856,"data_stored":383978479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790013,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623320,"kb_used_data":298692,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93748520,"statfs":{"total":96636764160,"available":95998484480,"internally_reserved":0,"allocated":305860608,"data_stored":305545411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783320,"kb_used_data":426820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93588520,"statfs":{"total":96636764160,"available":95834644480,"internally_reserved":0,"allocated":437063680,"data_stored":436792298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,3,1,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436441088,"data_stored":436404958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305508352,"data_stored":305428407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383082496,"data_stored":383042294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:20.628 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:20.628 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12807", 'reported_seq': 12992, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.188238+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:17.188238+0000', 'last_peered': '2024-09-11T01:20:17.188238+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:17.188238+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30437376, 'num_objects': 7431, 'num_object_clones': 0, 'num_object_copies': 14862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12807, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3366}, {'shards': '0,2', 'objects': 1435}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12612", 'reported_seq': 12746, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:15.472939+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:15.472939+0000', 'last_peered': '2024-09-11T01:20:15.472939+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:15.472939+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30445592, 'num_objects': 7434, 'num_object_clones': 0, 'num_object_copies': 14868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7434, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10673, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1417}, {'shards': '2', 'objects': 3456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12743", 'reported_seq': 12967, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.199948+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:17.199948+0000', 'last_peered': '2024-09-11T01:20:17.199948+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:17.199948+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30573027, 'num_objects': 7465, 'num_object_clones': 0, 'num_object_copies': 14930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4069, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7465, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9231, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3397}, {'shards': '0,2', 'objects': 1477}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13178", 'reported_seq': 13412, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.200382+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:17.200382+0000', 'last_peered': '2024-09-11T01:20:17.200382+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:17.200382+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31269830, 'num_objects': 7636, 'num_object_clones': 0, 'num_object_copies': 15272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4230, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7636, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 9931, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 53, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3406}, {'shards': '0,2', 'objects': 1514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'12960", 'reported_seq': 13187, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.129380+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:17.129380+0000', 'last_peered': '2024-09-11T01:20:17.129380+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:17.129380+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'2900", 'ondisk_log_start': "32'2900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31268864, 'num_objects': 7634, 'num_object_clones': 0, 'num_object_copies': 15268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4056, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9324, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3580}, {'shards': '0,2', 'objects': 1446}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.629 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12647", 'reported_seq': 12789, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:15.509944+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:15.509944+0000', 'last_peered': '2024-09-11T01:20:15.509944+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:15.509944+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2600", 'ondisk_log_start': "32'2600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30687715, 'num_objects': 7493, 'num_object_clones': 0, 'num_object_copies': 14986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7493, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10416, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1452}, {'shards': '2', 'objects': 3492}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'12940", 'reported_seq': 13076, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:15.581112+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:15.581112+0000', 'last_peered': '2024-09-11T01:20:15.581112+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:15.581112+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'2900", 'ondisk_log_start': "32'2900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31219712, 'num_objects': 7622, 'num_object_clones': 0, 'num_object_copies': 15244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10577, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1521}, {'shards': '2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'12883", 'reported_seq': 19464, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:17.208676+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:17.208676+0000', 'last_peered': '2024-09-11T01:20:17.208676+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:17.208676+0000', 'last_undegraded': '2024-09-11T01:20:17.208676+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30593024, 'num_objects': 7469, 'num_object_clones': 0, 'num_object_copies': 14938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 937, 'num_objects_unfound': 0, 'num_objects_dirty': 7469, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9573, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3228, 'num_bytes_recovered': 13217792, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6532}, {'shards': '0,2', 'objects': 937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'13027", 'reported_seq': 13247, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.196586+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:17.196586+0000', 'last_peered': '2024-09-11T01:20:17.196586+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:17.196586+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31207424, 'num_objects': 7619, 'num_object_clones': 0, 'num_object_copies': 15238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9420, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 45, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3472}, {'shards': '0,2', 'objects': 1492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'13022", 'reported_seq': 13267, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.185930+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:17.185930+0000', 'last_peered': '2024-09-11T01:20:17.185930+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:17.185930+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31039488, 'num_objects': 7578, 'num_object_clones': 0, 'num_object_copies': 15156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4092, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7578, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9557, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3488}, {'shards': '0,2', 'objects': 1428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:20.630 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12759", 'reported_seq': 12954, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:17.167549+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:17.167549+0000', 'last_peered': '2024-09-11T01:20:17.167549+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:17.167549+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30225350, 'num_objects': 7381, 'num_object_clones': 0, 'num_object_copies': 14762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4124, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7381, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9416, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3259}, {'shards': '0,2', 'objects': 1470}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:20.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:21.319 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:21.319 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40922/240496 objects degraded (17.016%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":421,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120248,"data_bytes":492487433,"bytes_used":2115842048,"bytes_avail":287794450432,"bytes_total":289910292480,"degraded_objects":40922,"degraded_total":240496,"degraded_ratio":0.17015667620251476,"misplaced_objects":937,"misplaced_total":240496,"misplaced_ratio":0.0038961146962943251,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":90759,"recovering_keys_per_sec":0,"num_objects_recovered":323,"num_bytes_recovered":1089536,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:21.336 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:21.337 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:21.866 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:21.866 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:21.888 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-11T01:20:20.543277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":492487433,"num_objects":120248,"num_object_clones":0,"num_object_copies":240496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40922,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":120248,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":164997,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7776,"num_bytes_recovered":30110198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160635,"ondisk_log_size":160635,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2066252,"kb_used_data":1100756,"kb_used_omap":12,"kb_used_meta":965427,"kb_avail":281049268,"statfs":{"total":289910292480,"available":287794450432,"internally_reserved":0,"allocated":1127174144,"data_stored":1126316188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":988597255},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,8,6,5,1,0,0,2,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18321408,"num_objects":-4473,"num_object_clones":0,"num_object_copies":-8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4473,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":323,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004677"},"pg_stats":[{"pgid":"2.f","version":"41'12807","reported_seq":12992,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.188238+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:17.188238+0000","last_peered":"2024-09-11T01:20:17.188238+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:17.188238+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30437376,"num_objects":7431,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12807,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3366},{"shards":"0,2","objects":1435}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'12918","reported_seq":12994,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:15.548904+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:15.548904+0000","last_peered":"2024-09-11T01:20:15.548904+0000","last_clean":"2024-09-11T01:20:15.548904+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:15.548904+0000","last_undegraded":"2024-09-11T01:20:15.548904+0000","last_fullsized":"2024-09-11T01:20:15.548904+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31146950,"num_objects":7606,"num_object_clones":0,"num_object_copies":15212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7606,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11016,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12612","reported_seq":12746,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.472939+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:15.472939+0000","last_peered":"2024-09-11T01:20:15.472939+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:15.472939+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30445592,"num_objects":7434,"num_object_clones":0,"num_object_copies":14868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10673,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1417},{"shards":"2","objects":3456}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12832","reported_seq":12873,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:15.534380+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:15.534380+0000","last_peered":"2024-09-11T01:20:15.534380+0000","last_clean":"2024-09-11T01:20:15.534380+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:15.534380+0000","last_undegraded":"2024-09-11T01:20:15.534380+0000","last_fullsized":"2024-09-11T01:20:15.534380+0000","mapping_epoch":31,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30842880,"num_objects":7530,"num_object_clones":0,"num_object_copies":15060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12832,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12821","reported_seq":12938,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:17.211794+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:17.211794+0000","last_peered":"2024-09-11T01:20:17.211794+0000","last_clean":"2024-09-11T01:20:17.211794+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:17.211794+0000","last_undegraded":"2024-09-11T01:20:17.211794+0000","last_fullsized":"2024-09-11T01:20:17.211794+0000","mapping_epoch":31,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30540259,"num_objects":7457,"num_object_clones":0,"num_object_copies":14914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7457,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9619,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12743","reported_seq":12967,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.199948+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:17.199948+0000","last_peered":"2024-09-11T01:20:17.199948+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:17.199948+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30573027,"num_objects":7465,"num_object_clones":0,"num_object_copies":14930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9231,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3397},{"shards":"0,2","objects":1477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13178","reported_seq":13412,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.200382+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:17.200382+0000","last_peered":"2024-09-11T01:20:17.200382+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:17.200382+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31269830,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9931,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":53,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3406},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'12919","reported_seq":13034,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:16.990653+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:16.990653+0000","last_peered":"2024-09-11T01:20:16.990653+0000","last_clean":"2024-09-11T01:20:16.990653+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:16.990653+0000","last_undegraded":"2024-09-11T01:20:16.990653+0000","last_fullsized":"2024-09-11T01:20:16.990653+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12864","reported_seq":21203,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.736682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:19.736682+0000","last_peered":"2024-09-11T01:20:19.736682+0000","last_clean":"2024-09-11T01:20:19.736682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:19.736682+0000","last_undegraded":"2024-09-11T01:20:19.736682+0000","last_fullsized":"2024-09-11T01:20:19.736682+0000","mapping_epoch":40,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30249443,"num_objects":7386,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10731,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'12960","reported_seq":13187,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.129380+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:17.129380+0000","last_peered":"2024-09-11T01:20:17.129380+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:17.129380+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9324,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3580},{"shards":"0,2","objects":1446}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12647","reported_seq":12789,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.509944+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:15.509944+0000","last_peered":"2024-09-11T01:20:15.509944+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:15.509944+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2600","ondisk_log_start":"32'2600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687715,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10416,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1452},{"shards":"2","objects":3492}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'12940","reported_seq":13076,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:15.581112+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:15.581112+0000","last_peered":"2024-09-11T01:20:15.581112+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:15.581112+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31219712,"num_objects":7622,"num_object_clones":0,"num_object_copies":15244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10577,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1521},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'12883","reported_seq":19464,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:17.208676+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:17.208676+0000","last_peered":"2024-09-11T01:20:17.208676+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:17.208676+0000","last_undegraded":"2024-09-11T01:20:17.208676+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30593024,"num_objects":7469,"num_object_clones":0,"num_object_copies":14938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":7469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9573,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3228,"num_bytes_recovered":13217792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6532},{"shards":"0,2","objects":937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13027","reported_seq":13247,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.196586+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:17.196586+0000","last_peered":"2024-09-11T01:20:17.196586+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:17.196586+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31207424,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9420,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":1492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13022","reported_seq":13267,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.185930+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:17.185930+0000","last_peered":"2024-09-11T01:20:17.185930+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:17.185930+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31039488,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4092,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3488},{"shards":"0,2","objects":1428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12759","reported_seq":12954,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.167549+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:17.167549+0000","last_peered":"2024-09-11T01:20:17.167549+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:17.167549+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30225350,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9416,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3259},{"shards":"0,2","objects":1470}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":492487414,"num_objects":120246,"num_object_clones":0,"num_object_copies":240492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40922,"num_objects_misplaced":937,"num_objects_unfound":0,"num_objects_dirty":120246,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":164995,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7775,"num_bytes_recovered":30110179,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125031936,"data_stored":1124875659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659612,"kb_used_data":375244,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93712228,"statfs":{"total":96636764160,"available":95961321472,"internally_reserved":0,"allocated":384249856,"data_stored":383978479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,0,0,0,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790013,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623320,"kb_used_data":298692,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93748520,"statfs":{"total":96636764160,"available":95998484480,"internally_reserved":0,"allocated":305860608,"data_stored":305545411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783320,"kb_used_data":426820,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93588520,"statfs":{"total":96636764160,"available":95834644480,"internally_reserved":0,"allocated":437063680,"data_stored":436792298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,3,1,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436441088,"data_stored":436404958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305508352,"data_stored":305428407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383082496,"data_stored":383042294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:24.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:25.451 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:25.452 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:25.474 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-11T01:20:24.547498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484320009,"num_objects":118254,"num_object_clones":0,"num_object_copies":236508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40908,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":118254,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":166991,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7960,"num_bytes_recovered":30806518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082044,"kb_used_data":1090172,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281033476,"statfs":{"total":289910292480,"available":287778279424,"internally_reserved":0,"allocated":1116336128,"data_stored":1115478172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1015598087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,8,2,3,0,0,0,0,2,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17690624,"num_objects":-4319,"num_object_clones":0,"num_object_copies":-8638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-317,"num_objects_unfound":0,"num_objects_dirty":-4319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":1298432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007175"},"pg_stats":[{"pgid":"2.f","version":"41'12949","reported_seq":13134,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.199787+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:22.199787+0000","last_peered":"2024-09-11T01:20:22.199787+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:22.199787+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":1293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13036","reported_seq":13112,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:20.498480+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:20.498480+0000","last_peered":"2024-09-11T01:20:20.498480+0000","last_clean":"2024-09-11T01:20:20.498480+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:20.498480+0000","last_undegraded":"2024-09-11T01:20:20.498480+0000","last_fullsized":"2024-09-11T01:20:20.498480+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30663622,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11134,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12760","reported_seq":12894,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:20.482534+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:20.482534+0000","last_peered":"2024-09-11T01:20:20.482534+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:20.482534+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839384,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10821,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1269},{"shards":"2","objects":3308}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12966","reported_seq":13007,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:20.551373+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:20.551373+0000","last_peered":"2024-09-11T01:20:20.551373+0000","last_clean":"2024-09-11T01:20:20.551373+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:20.551373+0000","last_undegraded":"2024-09-11T01:20:20.551373+0000","last_fullsized":"2024-09-11T01:20:20.551373+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12956","reported_seq":13073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.201638+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:22.201638+0000","last_peered":"2024-09-11T01:20:22.201638+0000","last_clean":"2024-09-11T01:20:22.201638+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:22.201638+0000","last_undegraded":"2024-09-11T01:20:22.201638+0000","last_fullsized":"2024-09-11T01:20:22.201638+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29987299,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9754,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12889","reported_seq":13115,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.245231+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:22.245231+0000","last_peered":"2024-09-11T01:20:22.245231+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:22.245231+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29975011,"num_objects":7319,"num_object_clones":0,"num_object_copies":14638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9377,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13334","reported_seq":13573,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.231042+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:22.231042+0000","last_peered":"2024-09-11T01:20:22.231042+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:22.231042+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30630854,"num_objects":7480,"num_object_clones":0,"num_object_copies":14960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10087,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":1358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13029","reported_seq":13142,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.244406+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:22.244406+0000","last_peered":"2024-09-11T01:20:22.244406+0000","last_clean":"2024-09-11T01:20:22.244406+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:22.244406+0000","last_undegraded":"2024-09-11T01:20:22.244406+0000","last_fullsized":"2024-09-11T01:20:22.244406+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12864","reported_seq":21203,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.736682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:19.736682+0000","last_peered":"2024-09-11T01:20:19.736682+0000","last_clean":"2024-09-11T01:20:19.736682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:19.736682+0000","last_undegraded":"2024-09-11T01:20:19.736682+0000","last_fullsized":"2024-09-11T01:20:19.736682+0000","mapping_epoch":40,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30249443,"num_objects":7386,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10731,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13094","reported_seq":13323,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.209761+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:22.209761+0000","last_peered":"2024-09-11T01:20:22.209761+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:22.209761+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9458,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":1312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12766","reported_seq":12918,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:20.575175+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:20.575175+0000","last_peered":"2024-09-11T01:20:20.575175+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:20.575175+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30200291,"num_objects":7374,"num_object_clones":0,"num_object_copies":14748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10535,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1333},{"shards":"2","objects":3378}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13060","reported_seq":13197,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:20.557084+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:20.557084+0000","last_peered":"2024-09-11T01:20:20.557084+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:20.557084+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1401},{"shards":"2","objects":3349}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13023","reported_seq":19945,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:22.269951+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:22.269951+0000","last_peered":"2024-09-11T01:20:22.269951+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:22.269951+0000","last_undegraded":"2024-09-11T01:20:22.269951+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9713,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3398,"num_bytes_recovered":13914112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6562},{"shards":"0,2","objects":767}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13163","reported_seq":13387,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.142188+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:22.142188+0000","last_peered":"2024-09-11T01:20:22.142188+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:22.142188+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9556,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3338},{"shards":"0,2","objects":1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13148","reported_seq":13397,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.186255+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:22.186255+0000","last_peered":"2024-09-11T01:20:22.186255+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:22.186255+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9683,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":1302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12889","reported_seq":13086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.117789+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:22.117789+0000","last_peered":"2024-09-11T01:20:22.117789+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:22.117789+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29692870,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9546,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3130},{"shards":"0,2","objects":1340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484319990,"num_objects":118252,"num_object_clones":0,"num_object_copies":236504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40908,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":118252,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":166989,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7959,"num_bytes_recovered":30806499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1114136576,"data_stored":1113980299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659468,"kb_used_data":370936,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93712372,"statfs":{"total":96636764160,"available":95961468928,"internally_reserved":0,"allocated":379838464,"data_stored":379567087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790013,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623320,"kb_used_data":298692,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93748520,"statfs":{"total":96636764160,"available":95998484480,"internally_reserved":0,"allocated":305860608,"data_stored":305545411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799256,"kb_used_data":420544,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93572584,"statfs":{"total":96636764160,"available":95818326016,"internally_reserved":0,"allocated":430637056,"data_stored":430365674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,0,1,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429838336,"data_stored":429802206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305508352,"data_stored":305428407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378789888,"data_stored":378749686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:25.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:26.035 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:26.036 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:26.063 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-11T01:20:24.547498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484320009,"num_objects":118254,"num_object_clones":0,"num_object_copies":236508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40908,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":118254,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":166991,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7960,"num_bytes_recovered":30806518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2082044,"kb_used_data":1090172,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281033476,"statfs":{"total":289910292480,"available":287778279424,"internally_reserved":0,"allocated":1116336128,"data_stored":1115478172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1015598087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,10,8,2,3,0,0,0,0,2,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17690624,"num_objects":-4319,"num_object_clones":0,"num_object_copies":-8638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-317,"num_objects_unfound":0,"num_objects_dirty":-4319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":1298432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007175"},"pg_stats":[{"pgid":"2.f","version":"41'12949","reported_seq":13134,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.199787+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:22.199787+0000","last_peered":"2024-09-11T01:20:22.199787+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:22.199787+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":1293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13036","reported_seq":13112,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:20.498480+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:20.498480+0000","last_peered":"2024-09-11T01:20:20.498480+0000","last_clean":"2024-09-11T01:20:20.498480+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:20.498480+0000","last_undegraded":"2024-09-11T01:20:20.498480+0000","last_fullsized":"2024-09-11T01:20:20.498480+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30663622,"num_objects":7488,"num_object_clones":0,"num_object_copies":14976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7488,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11134,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12760","reported_seq":12894,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:20.482534+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:20.482534+0000","last_peered":"2024-09-11T01:20:20.482534+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:20.482534+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839384,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10821,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1269},{"shards":"2","objects":3308}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'12966","reported_seq":13007,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:20.551373+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:20.551373+0000","last_peered":"2024-09-11T01:20:20.551373+0000","last_clean":"2024-09-11T01:20:20.551373+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:20.551373+0000","last_undegraded":"2024-09-11T01:20:20.551373+0000","last_fullsized":"2024-09-11T01:20:20.551373+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12956","reported_seq":13073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.201638+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:22.201638+0000","last_peered":"2024-09-11T01:20:22.201638+0000","last_clean":"2024-09-11T01:20:22.201638+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:22.201638+0000","last_undegraded":"2024-09-11T01:20:22.201638+0000","last_fullsized":"2024-09-11T01:20:22.201638+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29987299,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9754,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12889","reported_seq":13115,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.245231+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:22.245231+0000","last_peered":"2024-09-11T01:20:22.245231+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:22.245231+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29975011,"num_objects":7319,"num_object_clones":0,"num_object_copies":14638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9377,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13334","reported_seq":13573,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.231042+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:22.231042+0000","last_peered":"2024-09-11T01:20:22.231042+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:22.231042+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30630854,"num_objects":7480,"num_object_clones":0,"num_object_copies":14960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10087,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":1358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13029","reported_seq":13142,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.244406+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:22.244406+0000","last_peered":"2024-09-11T01:20:22.244406+0000","last_clean":"2024-09-11T01:20:22.244406+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:22.244406+0000","last_undegraded":"2024-09-11T01:20:22.244406+0000","last_fullsized":"2024-09-11T01:20:22.244406+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12864","reported_seq":21203,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.736682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:19.736682+0000","last_peered":"2024-09-11T01:20:19.736682+0000","last_clean":"2024-09-11T01:20:19.736682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:19.736682+0000","last_undegraded":"2024-09-11T01:20:19.736682+0000","last_fullsized":"2024-09-11T01:20:19.736682+0000","mapping_epoch":40,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30249443,"num_objects":7386,"num_object_clones":0,"num_object_copies":14772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10731,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13094","reported_seq":13323,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.209761+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:22.209761+0000","last_peered":"2024-09-11T01:20:22.209761+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:22.209761+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9458,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":1312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12766","reported_seq":12918,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:20.575175+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:20.575175+0000","last_peered":"2024-09-11T01:20:20.575175+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:20.575175+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2700","ondisk_log_start":"32'2700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30200291,"num_objects":7374,"num_object_clones":0,"num_object_copies":14748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7374,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10535,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1333},{"shards":"2","objects":3378}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13060","reported_seq":13197,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:20.557084+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:20.557084+0000","last_peered":"2024-09-11T01:20:20.557084+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:20.557084+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30728192,"num_objects":7502,"num_object_clones":0,"num_object_copies":15004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1401},{"shards":"2","objects":3349}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13023","reported_seq":19945,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:22.269951+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:22.269951+0000","last_peered":"2024-09-11T01:20:22.269951+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:22.269951+0000","last_undegraded":"2024-09-11T01:20:22.269951+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9713,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3398,"num_bytes_recovered":13914112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6562},{"shards":"0,2","objects":767}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13163","reported_seq":13387,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.142188+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:22.142188+0000","last_peered":"2024-09-11T01:20:22.142188+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:22.142188+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9556,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3338},{"shards":"0,2","objects":1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13148","reported_seq":13397,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.186255+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:22.186255+0000","last_peered":"2024-09-11T01:20:22.186255+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:22.186255+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9683,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":1302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12889","reported_seq":13086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.117789+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:22.117789+0000","last_peered":"2024-09-11T01:20:22.117789+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:22.117789+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29692870,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9546,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3130},{"shards":"0,2","objects":1340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484319990,"num_objects":118252,"num_object_clones":0,"num_object_copies":236504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40908,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":118252,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":166989,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7959,"num_bytes_recovered":30806499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1114136576,"data_stored":1113980299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659468,"kb_used_data":370936,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93712372,"statfs":{"total":96636764160,"available":95961468928,"internally_reserved":0,"allocated":379838464,"data_stored":379567087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,0,0,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790013,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623320,"kb_used_data":298692,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93748520,"statfs":{"total":96636764160,"available":95998484480,"internally_reserved":0,"allocated":305860608,"data_stored":305545411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799256,"kb_used_data":420544,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93572584,"statfs":{"total":96636764160,"available":95818326016,"internally_reserved":0,"allocated":430637056,"data_stored":430365674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,0,1,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429838336,"data_stored":429802206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305508352,"data_stored":305428407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378789888,"data_stored":378749686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:26.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:26.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'12949", 'reported_seq': 13134, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.199787+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:22.199787+0000', 'last_peered': '2024-09-11T01:20:22.199787+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:22.199787+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'2900", 'ondisk_log_start': "32'2900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29855744, 'num_objects': 7289, 'num_object_clones': 0, 'num_object_copies': 14578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12949, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3224}, {'shards': '0,2', 'objects': 1293}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12760", 'reported_seq': 12894, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:20.482534+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:20.482534+0000', 'last_peered': '2024-09-11T01:20:20.482534+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:20.482534+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29839384, 'num_objects': 7286, 'num_object_clones': 0, 'num_object_copies': 14572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7286, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10821, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1269}, {'shards': '2', 'objects': 3308}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'12889", 'reported_seq': 13115, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.245231+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:22.245231+0000', 'last_peered': '2024-09-11T01:20:22.245231+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:22.245231+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29975011, 'num_objects': 7319, 'num_object_clones': 0, 'num_object_copies': 14638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4068, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7319, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9377, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3252}, {'shards': '0,2', 'objects': 1331}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13334", 'reported_seq': 13573, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.231042+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:22.231042+0000', 'last_peered': '2024-09-11T01:20:22.231042+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:22.231042+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3300", 'ondisk_log_start': "32'3300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30630854, 'num_objects': 7480, 'num_object_clones': 0, 'num_object_copies': 14960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4228, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7480, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10087, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 55, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3252}, {'shards': '0,2', 'objects': 1358}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'13094", 'reported_seq': 13323, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.209761+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:22.209761+0000', 'last_peered': '2024-09-11T01:20:22.209761+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:22.209761+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30720000, 'num_objects': 7500, 'num_object_clones': 0, 'num_object_copies': 15000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4055, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9458, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3447}, {'shards': '0,2', 'objects': 1312}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:26.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12766", 'reported_seq': 12918, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:20.575175+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:20.575175+0000', 'last_peered': '2024-09-11T01:20:20.575175+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:20.575175+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2700", 'ondisk_log_start': "32'2700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30200291, 'num_objects': 7374, 'num_object_clones': 0, 'num_object_copies': 14748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3997, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7374, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10535, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 31, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1333}, {'shards': '2', 'objects': 3378}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'13060", 'reported_seq': 13197, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:20.557084+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:20.557084+0000', 'last_peered': '2024-09-11T01:20:20.557084+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:20.557084+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30728192, 'num_objects': 7502, 'num_object_clones': 0, 'num_object_copies': 15004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10697, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1401}, {'shards': '2', 'objects': 3349}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'13023", 'reported_seq': 19945, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:22.269951+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:22.269951+0000', 'last_peered': '2024-09-11T01:20:22.269951+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:22.269951+0000', 'last_undegraded': '2024-09-11T01:20:22.269951+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30019584, 'num_objects': 7329, 'num_object_clones': 0, 'num_object_copies': 14658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 767, 'num_objects_unfound': 0, 'num_objects_dirty': 7329, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9713, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3398, 'num_bytes_recovered': 13914112, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6562}, {'shards': '0,2', 'objects': 767}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'13163", 'reported_seq': 13387, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.142188+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:22.142188+0000', 'last_peered': '2024-09-11T01:20:22.142188+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:22.142188+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30650368, 'num_objects': 7483, 'num_object_clones': 0, 'num_object_copies': 14966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4148, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9556, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3338}, {'shards': '0,2', 'objects': 1356}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'13148", 'reported_seq': 13397, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.186255+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:22.186255+0000', 'last_peered': '2024-09-11T01:20:22.186255+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:22.186255+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30523392, 'num_objects': 7452, 'num_object_clones': 0, 'num_object_copies': 14904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4090, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7452, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9683, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3364}, {'shards': '0,2', 'objects': 1302}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:26.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'12889", 'reported_seq': 13086, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.117789+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:22.117789+0000', 'last_peered': '2024-09-11T01:20:22.117789+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:22.117789+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29692870, 'num_objects': 7251, 'num_object_clones': 0, 'num_object_copies': 14502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7251, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9546, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3130}, {'shards': '0,2', 'objects': 1340}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:26.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:26.771 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:26.771 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40908/236508 objects degraded (17.297%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":427,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118254,"data_bytes":484320009,"bytes_used":2132013056,"bytes_avail":287778279424,"bytes_total":289910292480,"degraded_objects":40908,"degraded_total":236508,"degraded_ratio":0.17296666497539195,"misplaced_objects":767,"misplaced_total":236508,"misplaced_ratio":0.0032430192636189897,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":108138,"recovering_keys_per_sec":0,"num_objects_recovered":360,"num_bytes_recovered":1298432,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":359},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:26.795 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:26.795 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:27.419 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:27.421 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:27.447 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-11T01:20:26.547855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":481071881,"num_objects":117461,"num_object_clones":0,"num_object_copies":234922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40906,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":117461,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":167784,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7962,"num_bytes_recovered":30806518,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2085800,"kb_used_data":1085612,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":281029720,"statfs":{"total":289910292480,"available":287774433280,"internally_reserved":0,"allocated":1111666688,"data_stored":1110808732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1024117767},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,4,5,3,0,0,0,0,0,7,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20938752,"num_objects":-5112,"num_object_clones":0,"num_object_copies":-10224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-45,"num_objects_misplaced":-317,"num_objects_unfound":0,"num_objects_dirty":-5112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5112,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":1298432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006751"},"pg_stats":[{"pgid":"2.f","version":"41'12949","reported_seq":13134,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.199787+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:22.199787+0000","last_peered":"2024-09-11T01:20:22.199787+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:22.199787+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29855744,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12949,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3224},{"shards":"0,2","objects":1293}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13173","reported_seq":13249,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:25.576026+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:25.576026+0000","last_peered":"2024-09-11T01:20:25.576026+0000","last_clean":"2024-09-11T01:20:25.576026+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:25.576026+0000","last_undegraded":"2024-09-11T01:20:25.576026+0000","last_fullsized":"2024-09-11T01:20:25.576026+0000","mapping_epoch":31,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30102470,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11271,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12876","reported_seq":13010,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.522522+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:25.522522+0000","last_peered":"2024-09-11T01:20:25.522522+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:25.522522+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364248,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10937,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1153},{"shards":"2","objects":3192}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13093","reported_seq":13134,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:25.519997+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:25.519997+0000","last_peered":"2024-09-11T01:20:25.519997+0000","last_clean":"2024-09-11T01:20:25.519997+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:25.519997+0000","last_undegraded":"2024-09-11T01:20:25.519997+0000","last_fullsized":"2024-09-11T01:20:25.519997+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29773824,"num_objects":7269,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13093,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'12956","reported_seq":13073,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.201638+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:22.201638+0000","last_peered":"2024-09-11T01:20:22.201638+0000","last_clean":"2024-09-11T01:20:22.201638+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:22.201638+0000","last_undegraded":"2024-09-11T01:20:22.201638+0000","last_fullsized":"2024-09-11T01:20:22.201638+0000","mapping_epoch":31,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29987299,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9754,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'12889","reported_seq":13115,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.245231+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:22.245231+0000","last_peered":"2024-09-11T01:20:22.245231+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:22.245231+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29975011,"num_objects":7319,"num_object_clones":0,"num_object_copies":14638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7319,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9377,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":1331}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13334","reported_seq":13573,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.231042+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:22.231042+0000","last_peered":"2024-09-11T01:20:22.231042+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:22.231042+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30630854,"num_objects":7480,"num_object_clones":0,"num_object_copies":14960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10087,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3252},{"shards":"0,2","objects":1358}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13029","reported_seq":13142,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.244406+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:22.244406+0000","last_peered":"2024-09-11T01:20:22.244406+0000","last_clean":"2024-09-11T01:20:22.244406+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:22.244406+0000","last_undegraded":"2024-09-11T01:20:22.244406+0000","last_fullsized":"2024-09-11T01:20:22.244406+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30289920,"num_objects":7395,"num_object_clones":0,"num_object_copies":14790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9982,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'12993","reported_seq":21332,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:24.721749+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:24.721749+0000","last_peered":"2024-09-11T01:20:24.721749+0000","last_clean":"2024-09-11T01:20:24.721749+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:24.721749+0000","last_undegraded":"2024-09-11T01:20:24.721749+0000","last_fullsized":"2024-09-11T01:20:24.721749+0000","mapping_epoch":40,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29721059,"num_objects":7257,"num_object_clones":0,"num_object_copies":14514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7257,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10860,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13094","reported_seq":13323,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.209761+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:22.209761+0000","last_peered":"2024-09-11T01:20:22.209761+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:22.209761+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30720000,"num_objects":7500,"num_object_clones":0,"num_object_copies":15000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9458,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3447},{"shards":"0,2","objects":1312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":93,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892515+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:17:37.892515+0000","last_peered":"2024-09-11T01:17:37.892515+0000","last_clean":"2024-09-11T01:17:37.892515+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:17:37.892515+0000","last_undegraded":"2024-09-11T01:17:37.892515+0000","last_fullsized":"2024-09-11T01:17:37.892515+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12902","reported_seq":13057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.555724+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:25.555724+0000","last_peered":"2024-09-11T01:20:25.555724+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:25.555724+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29643235,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3995,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10671,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1197},{"shards":"2","objects":3244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13208","reported_seq":13346,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.571940+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:25.571940+0000","last_peered":"2024-09-11T01:20:25.571940+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:25.571940+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30121984,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10845,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1253},{"shards":"2","objects":3201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13023","reported_seq":19945,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:22.269951+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:22.269951+0000","last_peered":"2024-09-11T01:20:22.269951+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:22.269951+0000","last_undegraded":"2024-09-11T01:20:22.269951+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9713,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3398,"num_bytes_recovered":13914112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6562},{"shards":"0,2","objects":767}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13163","reported_seq":13387,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.142188+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:22.142188+0000","last_peered":"2024-09-11T01:20:22.142188+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:22.142188+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4148,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9556,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3338},{"shards":"0,2","objects":1356}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13148","reported_seq":13397,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.186255+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:22.186255+0000","last_peered":"2024-09-11T01:20:22.186255+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:22.186255+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30523392,"num_objects":7452,"num_object_clones":0,"num_object_copies":14904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4090,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9683,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":1302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'12889","reported_seq":13086,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.117789+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:22.117789+0000","last_peered":"2024-09-11T01:20:22.117789+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:22.117789+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29692870,"num_objects":7251,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9546,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3130},{"shards":"0,2","objects":1340}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":481071862,"num_objects":117459,"num_object_clones":0,"num_object_copies":234918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40906,"num_objects_misplaced":767,"num_objects_unfound":0,"num_objects_dirty":117459,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":167782,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7961,"num_bytes_recovered":30806499,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1109463040,"data_stored":1109306763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659460,"kb_used_data":366768,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93712380,"statfs":{"total":96636764160,"available":95961477120,"internally_reserved":0,"allocated":375570432,"data_stored":375299055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,1,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790014,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627084,"kb_used_data":298300,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93744756,"statfs":{"total":96636764160,"available":95994630144,"internally_reserved":0,"allocated":305459200,"data_stored":305144003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":799256,"kb_used_data":420544,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93572584,"statfs":{"total":96636764160,"available":95818326016,"internally_reserved":0,"allocated":430637056,"data_stored":430365674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,0,1,0,0,0,0,0,7],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429838336,"data_stored":429802206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305135616,"data_stored":305055671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374489088,"data_stored":374448886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:30.450 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:31.002 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:31.005 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:31.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-11T01:20:30.549309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474870537,"num_objects":115947,"num_object_clones":0,"num_object_copies":231894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":611,"num_objects_unfound":0,"num_objects_dirty":115947,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":169298,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8144,"num_bytes_recovered":31445494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2074512,"kb_used_data":1078492,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281041008,"statfs":{"total":289910292480,"available":287785992192,"internally_reserved":0,"allocated":1104375808,"data_stored":1103517852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1019857927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,0,7,5,5,2,0,0,0,3,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18178048,"num_objects":-4438,"num_object_clones":0,"num_object_copies":-8876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-42,"num_objects_misplaced":-326,"num_objects_unfound":0,"num_objects_dirty":-4438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":1335296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006669"},"pg_stats":[{"pgid":"2.f","version":"41'13098","reported_seq":13289,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.271667+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:27.271667+0000","last_peered":"2024-09-11T01:20:27.271667+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:27.271667+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29245440,"num_objects":7140,"num_object_clones":0,"num_object_copies":14280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13098,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3078},{"shards":"0,2","objects":1144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13173","reported_seq":13249,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:25.576026+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:25.576026+0000","last_peered":"2024-09-11T01:20:25.576026+0000","last_clean":"2024-09-11T01:20:25.576026+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:25.576026+0000","last_undegraded":"2024-09-11T01:20:25.576026+0000","last_fullsized":"2024-09-11T01:20:25.576026+0000","mapping_epoch":31,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30102470,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11271,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12876","reported_seq":13010,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.522522+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:25.522522+0000","last_peered":"2024-09-11T01:20:25.522522+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:25.522522+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364248,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10937,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1153},{"shards":"2","objects":3192}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13093","reported_seq":13134,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:25.519997+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:25.519997+0000","last_peered":"2024-09-11T01:20:25.519997+0000","last_clean":"2024-09-11T01:20:25.519997+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:25.519997+0000","last_undegraded":"2024-09-11T01:20:25.519997+0000","last_fullsized":"2024-09-11T01:20:25.519997+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29773824,"num_objects":7269,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13093,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13100","reported_seq":13217,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.241777+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:27.241777+0000","last_peered":"2024-09-11T01:20:27.241777+0000","last_clean":"2024-09-11T01:20:27.241777+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:27.241777+0000","last_undegraded":"2024-09-11T01:20:27.241777+0000","last_fullsized":"2024-09-11T01:20:27.241777+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29397475,"num_objects":7178,"num_object_clones":0,"num_object_copies":14356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9898,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13021","reported_seq":13252,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.269850+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:27.269850+0000","last_peered":"2024-09-11T01:20:27.269850+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:27.269850+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29434339,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9509,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3123},{"shards":"0,2","objects":1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13475","reported_seq":13718,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.176629+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:27.176629+0000","last_peered":"2024-09-11T01:20:27.176629+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:27.176629+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30053318,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10228,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3113},{"shards":"0,2","objects":1217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13171","reported_seq":13286,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.245122+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:27.245122+0000","last_peered":"2024-09-11T01:20:27.245122+0000","last_clean":"2024-09-11T01:20:27.245122+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:27.245122+0000","last_undegraded":"2024-09-11T01:20:27.245122+0000","last_fullsized":"2024-09-11T01:20:27.245122+0000","mapping_epoch":31,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29708288,"num_objects":7253,"num_object_clones":0,"num_object_copies":14506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13138","reported_seq":21477,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.785223+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:29.785223+0000","last_peered":"2024-09-11T01:20:29.785223+0000","last_clean":"2024-09-11T01:20:29.785223+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:29.785223+0000","last_undegraded":"2024-09-11T01:20:29.785223+0000","last_fullsized":"2024-09-11T01:20:29.785223+0000","mapping_epoch":40,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29127139,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13223","reported_seq":13460,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.207915+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:27.207915+0000","last_peered":"2024-09-11T01:20:27.207915+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:27.207915+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4051,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9587,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3322},{"shards":"0,2","objects":1183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12902","reported_seq":13057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.555724+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:25.555724+0000","last_peered":"2024-09-11T01:20:25.555724+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:25.555724+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29643235,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3995,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10671,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1197},{"shards":"2","objects":3244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13208","reported_seq":13346,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.571940+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:25.571940+0000","last_peered":"2024-09-11T01:20:25.571940+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:25.571940+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30121984,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10845,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1253},{"shards":"2","objects":3201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13148","reported_seq":20380,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:27.279572+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:27.279572+0000","last_peered":"2024-09-11T01:20:27.279572+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:27.279572+0000","last_undegraded":"2024-09-11T01:20:27.279572+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29507584,"num_objects":7204,"num_object_clones":0,"num_object_copies":14408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":611,"num_objects_unfound":0,"num_objects_dirty":7204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9838,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":14553088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6593},{"shards":"0,2","objects":611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13293","reported_seq":13524,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.286671+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:27.286671+0000","last_peered":"2024-09-11T01:20:27.286671+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:27.286671+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9686,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3212},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13289","reported_seq":13546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.212658+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:27.212658+0000","last_peered":"2024-09-11T01:20:27.212658+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:27.212658+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29945856,"num_objects":7311,"num_object_clones":0,"num_object_copies":14622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9824,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3227},{"shards":"0,2","objects":1161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13025","reported_seq":13234,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.246688+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:27.246688+0000","last_peered":"2024-09-11T01:20:27.246688+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:27.246688+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29135814,"num_objects":7115,"num_object_clones":0,"num_object_copies":14230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9682,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3000},{"shards":"0,2","objects":1204}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474870518,"num_objects":115945,"num_object_clones":0,"num_object_copies":231890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":611,"num_objects_unfound":0,"num_objects_dirty":115945,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":169296,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8143,"num_bytes_recovered":31445475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1102221312,"data_stored":1102065035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659460,"kb_used_data":366768,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93712380,"statfs":{"total":96636764160,"available":95961477120,"internally_reserved":0,"allocated":375570432,"data_stored":375299055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,1,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790015,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618232,"kb_used_data":297768,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93753608,"statfs":{"total":96636764160,"available":96003694592,"internally_reserved":0,"allocated":304914432,"data_stored":304599235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":796820,"kb_used_data":413956,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93575020,"statfs":{"total":96636764160,"available":95820820480,"internally_reserved":0,"allocated":423890944,"data_stored":423619562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,3,2,2,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":423206912,"data_stored":423170782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304525312,"data_stored":304445367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374489088,"data_stored":374448886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:31.031 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:31.601 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:31.601 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:31.634 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-11T01:20:30.549309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474870537,"num_objects":115947,"num_object_clones":0,"num_object_copies":231894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":611,"num_objects_unfound":0,"num_objects_dirty":115947,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":169298,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8144,"num_bytes_recovered":31445494,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2074512,"kb_used_data":1078492,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":281041008,"statfs":{"total":289910292480,"available":287785992192,"internally_reserved":0,"allocated":1104375808,"data_stored":1103517852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1019857927},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,0,7,5,5,2,0,0,0,3,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18178048,"num_objects":-4438,"num_object_clones":0,"num_object_copies":-8876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-42,"num_objects_misplaced":-326,"num_objects_unfound":0,"num_objects_dirty":-4438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4438,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":1335296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006669"},"pg_stats":[{"pgid":"2.f","version":"41'13098","reported_seq":13289,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.271667+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:27.271667+0000","last_peered":"2024-09-11T01:20:27.271667+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:27.271667+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29245440,"num_objects":7140,"num_object_clones":0,"num_object_copies":14280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13098,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3078},{"shards":"0,2","objects":1144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13173","reported_seq":13249,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:25.576026+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:25.576026+0000","last_peered":"2024-09-11T01:20:25.576026+0000","last_clean":"2024-09-11T01:20:25.576026+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:25.576026+0000","last_undegraded":"2024-09-11T01:20:25.576026+0000","last_fullsized":"2024-09-11T01:20:25.576026+0000","mapping_epoch":31,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30102470,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11271,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'12876","reported_seq":13010,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.522522+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:25.522522+0000","last_peered":"2024-09-11T01:20:25.522522+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:25.522522+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2800","ondisk_log_start":"32'2800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364248,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10937,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1153},{"shards":"2","objects":3192}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13093","reported_seq":13134,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:25.519997+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:25.519997+0000","last_peered":"2024-09-11T01:20:25.519997+0000","last_clean":"2024-09-11T01:20:25.519997+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:25.519997+0000","last_undegraded":"2024-09-11T01:20:25.519997+0000","last_fullsized":"2024-09-11T01:20:25.519997+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29773824,"num_objects":7269,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13093,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13100","reported_seq":13217,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.241777+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:27.241777+0000","last_peered":"2024-09-11T01:20:27.241777+0000","last_clean":"2024-09-11T01:20:27.241777+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:27.241777+0000","last_undegraded":"2024-09-11T01:20:27.241777+0000","last_fullsized":"2024-09-11T01:20:27.241777+0000","mapping_epoch":31,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29397475,"num_objects":7178,"num_object_clones":0,"num_object_copies":14356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7178,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9898,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13021","reported_seq":13252,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.269850+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:27.269850+0000","last_peered":"2024-09-11T01:20:27.269850+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:27.269850+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29434339,"num_objects":7187,"num_object_clones":0,"num_object_copies":14374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7187,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9509,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3123},{"shards":"0,2","objects":1199}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13475","reported_seq":13718,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.176629+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:27.176629+0000","last_peered":"2024-09-11T01:20:27.176629+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:27.176629+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30053318,"num_objects":7339,"num_object_clones":0,"num_object_copies":14678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7339,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10228,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3113},{"shards":"0,2","objects":1217}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13171","reported_seq":13286,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.245122+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:27.245122+0000","last_peered":"2024-09-11T01:20:27.245122+0000","last_clean":"2024-09-11T01:20:27.245122+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:27.245122+0000","last_undegraded":"2024-09-11T01:20:27.245122+0000","last_fullsized":"2024-09-11T01:20:27.245122+0000","mapping_epoch":31,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29708288,"num_objects":7253,"num_object_clones":0,"num_object_copies":14506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13138","reported_seq":21477,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.785223+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:29.785223+0000","last_peered":"2024-09-11T01:20:29.785223+0000","last_clean":"2024-09-11T01:20:29.785223+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:29.785223+0000","last_undegraded":"2024-09-11T01:20:29.785223+0000","last_fullsized":"2024-09-11T01:20:29.785223+0000","mapping_epoch":40,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29127139,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13223","reported_seq":13460,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.207915+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:27.207915+0000","last_peered":"2024-09-11T01:20:27.207915+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:27.207915+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30191616,"num_objects":7371,"num_object_clones":0,"num_object_copies":14742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4051,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9587,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3322},{"shards":"0,2","objects":1183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'12902","reported_seq":13057,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.555724+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:25.555724+0000","last_peered":"2024-09-11T01:20:25.555724+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:25.555724+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29643235,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3995,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10671,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1197},{"shards":"2","objects":3244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13208","reported_seq":13346,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:25.571940+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:25.571940+0000","last_peered":"2024-09-11T01:20:25.571940+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:25.571940+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30121984,"num_objects":7354,"num_object_clones":0,"num_object_copies":14708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4155,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10845,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1253},{"shards":"2","objects":3201}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13148","reported_seq":20380,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:27.279572+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:27.279572+0000","last_peered":"2024-09-11T01:20:27.279572+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:27.279572+0000","last_undegraded":"2024-09-11T01:20:27.279572+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29507584,"num_objects":7204,"num_object_clones":0,"num_object_copies":14408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":611,"num_objects_unfound":0,"num_objects_dirty":7204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9838,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3554,"num_bytes_recovered":14553088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6593},{"shards":"0,2","objects":611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13293","reported_seq":13524,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.286671+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:27.286671+0000","last_peered":"2024-09-11T01:20:27.286671+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:27.286671+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30117888,"num_objects":7353,"num_object_clones":0,"num_object_copies":14706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4144,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9686,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3212},{"shards":"0,2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13289","reported_seq":13546,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.212658+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:27.212658+0000","last_peered":"2024-09-11T01:20:27.212658+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:27.212658+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29945856,"num_objects":7311,"num_object_clones":0,"num_object_copies":14622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9824,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3227},{"shards":"0,2","objects":1161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13025","reported_seq":13234,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.246688+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:27.246688+0000","last_peered":"2024-09-11T01:20:27.246688+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:27.246688+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29135814,"num_objects":7115,"num_object_clones":0,"num_object_copies":14230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4117,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7115,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9682,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3000},{"shards":"0,2","objects":1204}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474870518,"num_objects":115945,"num_object_clones":0,"num_object_copies":231890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40880,"num_objects_misplaced":611,"num_objects_unfound":0,"num_objects_dirty":115945,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":169296,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8143,"num_bytes_recovered":31445475,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1102221312,"data_stored":1102065035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659460,"kb_used_data":366768,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93712380,"statfs":{"total":96636764160,"available":95961477120,"internally_reserved":0,"allocated":375570432,"data_stored":375299055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,1,0,0,0,0,0,0,0,2],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790015,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618232,"kb_used_data":297768,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93753608,"statfs":{"total":96636764160,"available":96003694592,"internally_reserved":0,"allocated":304914432,"data_stored":304599235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":796820,"kb_used_data":413956,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93575020,"statfs":{"total":96636764160,"available":95820820480,"internally_reserved":0,"allocated":423890944,"data_stored":423619562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,3,2,2,0,0,0,0,3],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":423206912,"data_stored":423170782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304525312,"data_stored":304445367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374489088,"data_stored":374448886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:31.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:31.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'13098", 'reported_seq': 13289, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.271667+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:27.271667+0000', 'last_peered': '2024-09-11T01:20:27.271667+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:27.271667+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29245440, 'num_objects': 7140, 'num_object_clones': 0, 'num_object_copies': 14280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4062, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7140, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13098, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3078}, {'shards': '0,2', 'objects': 1144}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:31.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'12876", 'reported_seq': 13010, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:25.522522+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:25.522522+0000', 'last_peered': '2024-09-11T01:20:25.522522+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:25.522522+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'2800", 'ondisk_log_start': "32'2800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29364248, 'num_objects': 7170, 'num_object_clones': 0, 'num_object_copies': 14340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3979, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7170, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10937, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1153}, {'shards': '2', 'objects': 3192}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:31.635 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:31.635 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'13021", 'reported_seq': 13252, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.269850+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:27.269850+0000', 'last_peered': '2024-09-11T01:20:27.269850+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:27.269850+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29434339, 'num_objects': 7187, 'num_object_clones': 0, 'num_object_copies': 14374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7187, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9509, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3123}, {'shards': '0,2', 'objects': 1199}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13475", 'reported_seq': 13718, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.176629+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:27.176629+0000', 'last_peered': '2024-09-11T01:20:27.176629+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:27.176629+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30053318, 'num_objects': 7339, 'num_object_clones': 0, 'num_object_copies': 14678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4226, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7339, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10228, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 57, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3113}, {'shards': '0,2', 'objects': 1217}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'13223", 'reported_seq': 13460, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.207915+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:27.207915+0000', 'last_peered': '2024-09-11T01:20:27.207915+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:27.207915+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30191616, 'num_objects': 7371, 'num_object_clones': 0, 'num_object_copies': 14742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4051, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9587, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3322}, {'shards': '0,2', 'objects': 1183}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'12902", 'reported_seq': 13057, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:25.555724+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:25.555724+0000', 'last_peered': '2024-09-11T01:20:25.555724+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:25.555724+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'2900", 'ondisk_log_start': "32'2900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29643235, 'num_objects': 7238, 'num_object_clones': 0, 'num_object_copies': 14476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3995, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7238, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10671, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 33, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1197}, {'shards': '2', 'objects': 3244}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'13208", 'reported_seq': 13346, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:25.571940+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:25.571940+0000', 'last_peered': '2024-09-11T01:20:25.571940+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:25.571940+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30121984, 'num_objects': 7354, 'num_object_clones': 0, 'num_object_copies': 14708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4155, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10845, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 24, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1253}, {'shards': '2', 'objects': 3201}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'13148", 'reported_seq': 20380, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:27.279572+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:27.279572+0000', 'last_peered': '2024-09-11T01:20:27.279572+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:27.279572+0000', 'last_undegraded': '2024-09-11T01:20:27.279572+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29507584, 'num_objects': 7204, 'num_object_clones': 0, 'num_object_copies': 14408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 611, 'num_objects_unfound': 0, 'num_objects_dirty': 7204, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9838, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3554, 'num_bytes_recovered': 14553088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6593}, {'shards': '0,2', 'objects': 611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:31.636 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'13293", 'reported_seq': 13524, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.286671+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:27.286671+0000', 'last_peered': '2024-09-11T01:20:27.286671+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:27.286671+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30117888, 'num_objects': 7353, 'num_object_clones': 0, 'num_object_copies': 14706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4144, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7353, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9686, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3212}, {'shards': '0,2', 'objects': 1226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:31.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'13289", 'reported_seq': 13546, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.212658+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:27.212658+0000', 'last_peered': '2024-09-11T01:20:27.212658+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:27.212658+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29945856, 'num_objects': 7311, 'num_object_clones': 0, 'num_object_copies': 14622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4086, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9824, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 64, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3227}, {'shards': '0,2', 'objects': 1161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.637 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:31.637 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'13025", 'reported_seq': 13234, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.246688+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:27.246688+0000', 'last_peered': '2024-09-11T01:20:27.246688+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:27.246688+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'3000", 'ondisk_log_start': "32'3000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29135814, 'num_objects': 7115, 'num_object_clones': 0, 'num_object_copies': 14230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4117, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7115, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9682, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3000}, {'shards': '0,2', 'objects': 1204}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.637 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:32.338 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:32.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40880/231894 objects degraded (17.629%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115947,"data_bytes":474870537,"bytes_used":2124300288,"bytes_avail":287785992192,"bytes_total":289910292480,"degraded_objects":40880,"degraded_total":231894,"degraded_ratio":0.17628744167593816,"misplaced_objects":611,"misplaced_total":231894,"misplaced_ratio":0.0026348245318981948,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":111212,"recovering_keys_per_sec":0,"num_objects_recovered":368,"num_bytes_recovered":1335296,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:32.365 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:32.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:32.954 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:32.954 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:32.975 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-11T01:20:32.550138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466285321,"num_objects":113851,"num_object_clones":0,"num_object_copies":227702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40861,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113851,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":171394,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8307,"num_bytes_recovered":32035318,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059268,"kb_used_data":1067408,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281056252,"statfs":{"total":289910292480,"available":287801602048,"internally_reserved":0,"allocated":1093025792,"data_stored":1092167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1015598087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,7,9,3,2,0,0,3,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26202112,"num_objects":-6397,"num_object_clones":0,"num_object_copies":-12794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-61,"num_objects_misplaced":-470,"num_objects_unfound":0,"num_objects_dirty":-6397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6397,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":1925120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006861"},"pg_stats":[{"pgid":"2.f","version":"41'13216","reported_seq":13407,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.315355+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:32.315355+0000","last_peered":"2024-09-11T01:20:32.315355+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:32.315355+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":1026}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13296","reported_seq":13372,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.526085+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:30.526085+0000","last_peered":"2024-09-11T01:20:30.526085+0000","last_clean":"2024-09-11T01:20:30.526085+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:30.526085+0000","last_undegraded":"2024-09-11T01:20:30.526085+0000","last_fullsized":"2024-09-11T01:20:30.526085+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598662,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11394,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13000","reported_seq":13136,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:30.568145+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:30.568145+0000","last_peered":"2024-09-11T01:20:30.568145+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:30.568145+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856344,"num_objects":7046,"num_object_clones":0,"num_object_copies":14092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11061,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1029},{"shards":"2","objects":3069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13223","reported_seq":13264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.441981+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:30.441981+0000","last_peered":"2024-09-11T01:20:30.441981+0000","last_clean":"2024-09-11T01:20:30.441981+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:30.441981+0000","last_undegraded":"2024-09-11T01:20:30.441981+0000","last_fullsized":"2024-09-11T01:20:30.441981+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29241344,"num_objects":7139,"num_object_clones":0,"num_object_copies":14278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13223,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13228","reported_seq":13345,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.287820+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:32.287820+0000","last_peered":"2024-09-11T01:20:32.287820+0000","last_clean":"2024-09-11T01:20:32.287820+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:32.287820+0000","last_undegraded":"2024-09-11T01:20:32.287820+0000","last_fullsized":"2024-09-11T01:20:32.287820+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10026,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13163","reported_seq":13394,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.305530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:32.305530+0000","last_peered":"2024-09-11T01:20:32.305530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:32.305530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852707,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9651,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2981},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13620","reported_seq":13863,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.272649+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:32.272649+0000","last_peered":"2024-09-11T01:20:32.272649+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:32.272649+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29459398,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":1072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13330","reported_seq":13445,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.240046+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:32.240046+0000","last_peered":"2024-09-11T01:20:32.240046+0000","last_clean":"2024-09-11T01:20:32.240046+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:32.240046+0000","last_undegraded":"2024-09-11T01:20:32.240046+0000","last_fullsized":"2024-09-11T01:20:32.240046+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13138","reported_seq":21477,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.785223+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:29.785223+0000","last_peered":"2024-09-11T01:20:29.785223+0000","last_clean":"2024-09-11T01:20:29.785223+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:29.785223+0000","last_undegraded":"2024-09-11T01:20:29.785223+0000","last_fullsized":"2024-09-11T01:20:29.785223+0000","mapping_epoch":40,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29127139,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13363","reported_seq":13606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.214238+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:32.214238+0000","last_peered":"2024-09-11T01:20:32.214238+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:32.214238+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618176,"num_objects":7231,"num_object_clones":0,"num_object_copies":14462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3185},{"shards":"0,2","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13031","reported_seq":13189,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:30.574783+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:30.574783+0000","last_peered":"2024-09-11T01:20:30.574783+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:30.574783+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29114851,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10800,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1068},{"shards":"2","objects":3116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13370","reported_seq":13510,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:30.483142+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:30.483142+0000","last_peered":"2024-09-11T01:20:30.483142+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:30.483142+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29458432,"num_objects":7192,"num_object_clones":0,"num_object_copies":14384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1091},{"shards":"2","objects":3040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13317","reported_seq":20840,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:32.290807+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:32.290807+0000","last_peered":"2024-09-11T01:20:32.290807+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:32.290807+0000","last_undegraded":"2024-09-11T01:20:32.290807+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13432","reported_seq":13674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.165967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:32.165967+0000","last_peered":"2024-09-11T01:20:32.165967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:32.165967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9825,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3078},{"shards":"0,2","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13430","reported_seq":13695,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.167366+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:32.167366+0000","last_peered":"2024-09-11T01:20:32.167366+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:32.167366+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368320,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9965,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3090},{"shards":"0,2","objects":1020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13172","reported_seq":13386,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.293027+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:32.293027+0000","last_peered":"2024-09-11T01:20:32.293027+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:32.293027+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28533702,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9829,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2856},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466285302,"num_objects":113849,"num_object_clones":0,"num_object_copies":227698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40861,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113849,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":171392,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8306,"num_bytes_recovered":32035299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1090764800,"data_stored":1090608523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659216,"kb_used_data":362368,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93712624,"statfs":{"total":96636764160,"available":95961726976,"internally_reserved":0,"allocated":371064832,"data_stored":370793455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,1,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790015,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618232,"kb_used_data":297768,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93753608,"statfs":{"total":96636764160,"available":96003694592,"internally_reserved":0,"allocated":304914432,"data_stored":304599235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781820,"kb_used_data":407272,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93590020,"statfs":{"total":96636764160,"available":95836180480,"internally_reserved":0,"allocated":417046528,"data_stored":416775146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416313344,"data_stored":416277214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304525312,"data_stored":304445367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369926144,"data_stored":369885942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:35.977 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:36.513 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:36.516 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:36.541 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-11T01:20:34.550990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":466285321,"num_objects":113851,"num_object_clones":0,"num_object_copies":227702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40861,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113851,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":171394,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8307,"num_bytes_recovered":32035318,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059268,"kb_used_data":1067408,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281056252,"statfs":{"total":289910292480,"available":287801602048,"internally_reserved":0,"allocated":1093025792,"data_stored":1092167836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1015598087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,0,7,9,3,2,0,0,3,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18034688,"num_objects":-4403,"num_object_clones":0,"num_object_copies":-8806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-47,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":-4403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4403,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":347,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004154"},"pg_stats":[{"pgid":"2.f","version":"41'13216","reported_seq":13407,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.315355+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:32.315355+0000","last_peered":"2024-09-11T01:20:32.315355+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:32.315355+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":1026}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13296","reported_seq":13372,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.526085+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:30.526085+0000","last_peered":"2024-09-11T01:20:30.526085+0000","last_clean":"2024-09-11T01:20:30.526085+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:30.526085+0000","last_undegraded":"2024-09-11T01:20:30.526085+0000","last_fullsized":"2024-09-11T01:20:30.526085+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598662,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11394,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13000","reported_seq":13136,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:30.568145+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:30.568145+0000","last_peered":"2024-09-11T01:20:30.568145+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:30.568145+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'2900","ondisk_log_start":"32'2900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28856344,"num_objects":7046,"num_object_clones":0,"num_object_copies":14092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3978,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11061,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1029},{"shards":"2","objects":3069}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13223","reported_seq":13264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.441981+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:30.441981+0000","last_peered":"2024-09-11T01:20:30.441981+0000","last_clean":"2024-09-11T01:20:30.441981+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:30.441981+0000","last_undegraded":"2024-09-11T01:20:30.441981+0000","last_fullsized":"2024-09-11T01:20:30.441981+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29241344,"num_objects":7139,"num_object_clones":0,"num_object_copies":14278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13223,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13228","reported_seq":13345,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.287820+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:32.287820+0000","last_peered":"2024-09-11T01:20:32.287820+0000","last_clean":"2024-09-11T01:20:32.287820+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:32.287820+0000","last_undegraded":"2024-09-11T01:20:32.287820+0000","last_fullsized":"2024-09-11T01:20:32.287820+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10026,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13163","reported_seq":13394,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.305530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:32.305530+0000","last_peered":"2024-09-11T01:20:32.305530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:32.305530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852707,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9651,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2981},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13620","reported_seq":13863,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.272649+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:32.272649+0000","last_peered":"2024-09-11T01:20:32.272649+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:32.272649+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29459398,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":1072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13330","reported_seq":13445,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.240046+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:32.240046+0000","last_peered":"2024-09-11T01:20:32.240046+0000","last_clean":"2024-09-11T01:20:32.240046+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:32.240046+0000","last_undegraded":"2024-09-11T01:20:32.240046+0000","last_fullsized":"2024-09-11T01:20:32.240046+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13138","reported_seq":21477,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.785223+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:29.785223+0000","last_peered":"2024-09-11T01:20:29.785223+0000","last_clean":"2024-09-11T01:20:29.785223+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:29.785223+0000","last_undegraded":"2024-09-11T01:20:29.785223+0000","last_fullsized":"2024-09-11T01:20:29.785223+0000","mapping_epoch":40,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29127139,"num_objects":7112,"num_object_clones":0,"num_object_copies":14224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13363","reported_seq":13606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.214238+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:32.214238+0000","last_peered":"2024-09-11T01:20:32.214238+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:32.214238+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618176,"num_objects":7231,"num_object_clones":0,"num_object_copies":14462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3185},{"shards":"0,2","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13031","reported_seq":13189,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:30.574783+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:30.574783+0000","last_peered":"2024-09-11T01:20:30.574783+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:30.574783+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3000","ondisk_log_start":"32'3000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29114851,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10800,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1068},{"shards":"2","objects":3116}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13370","reported_seq":13510,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:30.483142+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:30.483142+0000","last_peered":"2024-09-11T01:20:30.483142+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:30.483142+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29458432,"num_objects":7192,"num_object_clones":0,"num_object_copies":14384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1091},{"shards":"2","objects":3040}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13317","reported_seq":20840,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:32.290807+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:32.290807+0000","last_peered":"2024-09-11T01:20:32.290807+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:32.290807+0000","last_undegraded":"2024-09-11T01:20:32.290807+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13432","reported_seq":13674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.165967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:32.165967+0000","last_peered":"2024-09-11T01:20:32.165967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:32.165967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9825,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3078},{"shards":"0,2","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13430","reported_seq":13695,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.167366+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:32.167366+0000","last_peered":"2024-09-11T01:20:32.167366+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:32.167366+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368320,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9965,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3090},{"shards":"0,2","objects":1020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13172","reported_seq":13386,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.293027+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:32.293027+0000","last_peered":"2024-09-11T01:20:32.293027+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:32.293027+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28533702,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9829,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2856},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":466285302,"num_objects":113849,"num_object_clones":0,"num_object_copies":227698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40861,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113849,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":171392,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8306,"num_bytes_recovered":32035299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1090764800,"data_stored":1090608523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160729,"ondisk_log_size":160729,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659216,"kb_used_data":362368,"kb_used_omap":4,"kb_used_meta":296827,"kb_avail":93712624,"statfs":{"total":96636764160,"available":95961726976,"internally_reserved":0,"allocated":371064832,"data_stored":370793455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":303951534},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,1,0,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790015,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":618232,"kb_used_data":297768,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93753608,"statfs":{"total":96636764160,"available":96003694592,"internally_reserved":0,"allocated":304914432,"data_stored":304599235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781820,"kb_used_data":407272,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93590020,"statfs":{"total":96636764160,"available":95836180480,"internally_reserved":0,"allocated":417046528,"data_stored":416775146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416313344,"data_stored":416277214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304525312,"data_stored":304445367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369926144,"data_stored":369885942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:36.543 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:37.114 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:37.115 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:37.144 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-11T01:20:36.551886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":462955273,"num_objects":113038,"num_object_clones":0,"num_object_copies":226076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40853,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113038,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":172207,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8315,"num_bytes_recovered":32035318,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160645,"ondisk_log_size":160645,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046000,"kb_used_data":1062460,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281069520,"statfs":{"total":289910292480,"available":287815188480,"internally_reserved":0,"allocated":1087959040,"data_stored":1087101084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1007078407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,10,8,0,0,0,0,3,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21364736,"num_objects":-5216,"num_object_clones":0,"num_object_copies":-10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-55,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":-5216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004388"},"pg_stats":[{"pgid":"2.f","version":"41'13216","reported_seq":13407,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.315355+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:32.315355+0000","last_peered":"2024-09-11T01:20:32.315355+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:32.315355+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":1026}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13430","reported_seq":13506,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.496189+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:35.496189+0000","last_peered":"2024-09-11T01:20:35.496189+0000","last_clean":"2024-09-11T01:20:35.496189+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:35.496189+0000","last_undegraded":"2024-09-11T01:20:35.496189+0000","last_fullsized":"2024-09-11T01:20:35.496189+0000","mapping_epoch":31,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29049798,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11528,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13152","reported_seq":13294,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.556891+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:35.556891+0000","last_peered":"2024-09-11T01:20:35.556891+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:35.556891+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233752,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":877},{"shards":"2","objects":2920}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13343","reported_seq":13384,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.515606+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:35.515606+0000","last_peered":"2024-09-11T01:20:35.515606+0000","last_clean":"2024-09-11T01:20:35.515606+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:35.515606+0000","last_undegraded":"2024-09-11T01:20:35.515606+0000","last_fullsized":"2024-09-11T01:20:35.515606+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28749824,"num_objects":7019,"num_object_clones":0,"num_object_copies":14038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13228","reported_seq":13345,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.287820+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:32.287820+0000","last_peered":"2024-09-11T01:20:32.287820+0000","last_clean":"2024-09-11T01:20:32.287820+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:32.287820+0000","last_undegraded":"2024-09-11T01:20:32.287820+0000","last_fullsized":"2024-09-11T01:20:32.287820+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10026,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13163","reported_seq":13394,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.305530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:32.305530+0000","last_peered":"2024-09-11T01:20:32.305530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:32.305530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852707,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9651,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2981},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13620","reported_seq":13863,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.272649+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:32.272649+0000","last_peered":"2024-09-11T01:20:32.272649+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:32.272649+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29459398,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":1072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13330","reported_seq":13445,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.240046+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:32.240046+0000","last_peered":"2024-09-11T01:20:32.240046+0000","last_clean":"2024-09-11T01:20:32.240046+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:32.240046+0000","last_undegraded":"2024-09-11T01:20:32.240046+0000","last_fullsized":"2024-09-11T01:20:32.240046+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13288","reported_seq":21627,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.767963+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:34.767963+0000","last_peered":"2024-09-11T01:20:34.767963+0000","last_clean":"2024-09-11T01:20:34.767963+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:34.767963+0000","last_undegraded":"2024-09-11T01:20:34.767963+0000","last_fullsized":"2024-09-11T01:20:34.767963+0000","mapping_epoch":40,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512739,"num_objects":6962,"num_object_clones":0,"num_object_copies":13924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11155,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13363","reported_seq":13606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.214238+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:32.214238+0000","last_peered":"2024-09-11T01:20:32.214238+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:32.214238+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618176,"num_objects":7231,"num_object_clones":0,"num_object_copies":14462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3185},{"shards":"0,2","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13155","reported_seq":13323,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.563725+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:35.563725+0000","last_peered":"2024-09-11T01:20:35.563725+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:35.563725+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606947,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10924,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":944},{"shards":"2","objects":2997}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13503","reported_seq":13643,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.505301+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:35.505301+0000","last_peered":"2024-09-11T01:20:35.505301+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:35.505301+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11140,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":958},{"shards":"2","objects":2907}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13317","reported_seq":20840,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:32.290807+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:32.290807+0000","last_peered":"2024-09-11T01:20:32.290807+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:32.290807+0000","last_undegraded":"2024-09-11T01:20:32.290807+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13432","reported_seq":13674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.165967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:32.165967+0000","last_peered":"2024-09-11T01:20:32.165967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:32.165967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9825,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3078},{"shards":"0,2","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13430","reported_seq":13695,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.167366+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:32.167366+0000","last_peered":"2024-09-11T01:20:32.167366+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:32.167366+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368320,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9965,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3090},{"shards":"0,2","objects":1020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13172","reported_seq":13386,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.293027+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:32.293027+0000","last_peered":"2024-09-11T01:20:32.293027+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:32.293027+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28533702,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9829,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2856},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":462955254,"num_objects":113036,"num_object_clones":0,"num_object_copies":226072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40853,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113036,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":172205,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8314,"num_bytes_recovered":32035299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085788160,"data_stored":1085631883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160642,"ondisk_log_size":160642,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642312,"kb_used_data":357944,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93729528,"statfs":{"total":96636764160,"available":95979036672,"internally_reserved":0,"allocated":366534656,"data_stored":366263279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,1,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790016,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621868,"kb_used_data":297244,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93749972,"statfs":{"total":96636764160,"available":95999971328,"internally_reserved":0,"allocated":304377856,"data_stored":304062659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781820,"kb_used_data":407272,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93590020,"statfs":{"total":96636764160,"available":95836180480,"internally_reserved":0,"allocated":417046528,"data_stored":416775146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416313344,"data_stored":416277214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304001024,"data_stored":303921079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365473792,"data_stored":365433590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:37.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'13216", 'reported_seq': 13407, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.315355+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:32.315355+0000', 'last_peered': '2024-09-11T01:20:32.315355+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:32.315355+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28762112, 'num_objects': 7022, 'num_object_clones': 0, 'num_object_copies': 14044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4062, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13216, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2960}, {'shards': '0,2', 'objects': 1026}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'13152", 'reported_seq': 13294, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:35.556891+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:35.556891+0000', 'last_peered': '2024-09-11T01:20:35.556891+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:35.556891+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28233752, 'num_objects': 6894, 'num_object_clones': 0, 'num_object_copies': 13788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3975, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6894, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11213, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 877}, {'shards': '2', 'objects': 2920}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'13163", 'reported_seq': 13394, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.305530+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:32.305530+0000', 'last_peered': '2024-09-11T01:20:32.305530+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:32.305530+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28852707, 'num_objects': 7045, 'num_object_clones': 0, 'num_object_copies': 14090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4065, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7045, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9651, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 51, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2981}, {'shards': '0,2', 'objects': 1057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13620", 'reported_seq': 13863, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.272649+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:32.272649+0000', 'last_peered': '2024-09-11T01:20:32.272649+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:32.272649+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29459398, 'num_objects': 7194, 'num_object_clones': 0, 'num_object_copies': 14388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4225, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7194, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10373, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 58, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2969}, {'shards': '0,2', 'objects': 1072}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:37.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'13363", 'reported_seq': 13606, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.214238+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:32.214238+0000', 'last_peered': '2024-09-11T01:20:32.214238+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:32.214238+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'3300", 'ondisk_log_start': "32'3300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29618176, 'num_objects': 7231, 'num_object_clones': 0, 'num_object_copies': 14462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9727, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3185}, {'shards': '0,2', 'objects': 1043}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'13155", 'reported_seq': 13323, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:35.563725+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:35.563725+0000', 'last_peered': '2024-09-11T01:20:35.563725+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:35.563725+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28606947, 'num_objects': 6985, 'num_object_clones': 0, 'num_object_copies': 13970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3989, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6985, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10924, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 944}, {'shards': '2', 'objects': 2997}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'13503", 'reported_seq': 13643, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:35.505301+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:35.505301+0000', 'last_peered': '2024-09-11T01:20:35.505301+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:35.505301+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'3500", 'ondisk_log_start': "32'3500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28913664, 'num_objects': 7059, 'num_object_clones': 0, 'num_object_copies': 14118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11140, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 958}, {'shards': '2', 'objects': 2907}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'13317", 'reported_seq': 20840, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:32.290807+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:32.290807+0000', 'last_peered': '2024-09-11T01:20:32.290807+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:32.290807+0000', 'last_undegraded': '2024-09-11T01:20:32.290807+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'3300", 'ondisk_log_start': "32'3300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28815360, 'num_objects': 7035, 'num_object_clones': 0, 'num_object_copies': 14070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 467, 'num_objects_unfound': 0, 'num_objects_dirty': 7035, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10007, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3698, 'num_bytes_recovered': 15142912, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6568}, {'shards': '0,2', 'objects': 467}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'13432", 'reported_seq': 13674, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.165967+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:32.165967+0000', 'last_peered': '2024-09-11T01:20:32.165967+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:32.165967+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29548544, 'num_objects': 7214, 'num_object_clones': 0, 'num_object_copies': 14428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4139, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9825, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 56, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3078}, {'shards': '0,2', 'objects': 1087}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'13430", 'reported_seq': 13695, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.167366+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:32.167366+0000', 'last_peered': '2024-09-11T01:20:32.167366+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:32.167366+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29368320, 'num_objects': 7170, 'num_object_clones': 0, 'num_object_copies': 14340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4082, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9965, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3090}, {'shards': '0,2', 'objects': 1020}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.150 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:37.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'13172", 'reported_seq': 13386, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.293027+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:32.293027+0000', 'last_peered': '2024-09-11T01:20:32.293027+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:32.293027+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'3100", 'ondisk_log_start': "32'3100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28533702, 'num_objects': 6968, 'num_object_clones': 0, 'num_object_copies': 13936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6968, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9829, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2856}, {'shards': '0,2', 'objects': 1057}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:37.151 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:37.865 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:37.865 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40853/226076 objects degraded (18.070%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":438,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":113038,"data_bytes":462955273,"bytes_used":2095104000,"bytes_avail":287815188480,"bytes_total":289910292480,"degraded_objects":40853,"degraded_total":226076,"degraded_ratio":0.18070471876714028,"misplaced_objects":467,"misplaced_total":226076,"misplaced_ratio":0.0020656770289637113,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":102362,"recovering_keys_per_sec":0,"num_objects_recovered":355,"num_bytes_recovered":1228800,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:37.893 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:37.894 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:38.466 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:38.467 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:38.489 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-11T01:20:36.551886+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":462955273,"num_objects":113038,"num_object_clones":0,"num_object_copies":226076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40853,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113038,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":172207,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8315,"num_bytes_recovered":32035318,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160645,"ondisk_log_size":160645,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046000,"kb_used_data":1062460,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":281069520,"statfs":{"total":289910292480,"available":287815188480,"internally_reserved":0,"allocated":1087959040,"data_stored":1087101084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1007078407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,10,8,0,0,0,0,3,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21364736,"num_objects":-5216,"num_object_clones":0,"num_object_copies":-10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-55,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":-5216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1228800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004388"},"pg_stats":[{"pgid":"2.f","version":"41'13216","reported_seq":13407,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.315355+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:32.315355+0000","last_peered":"2024-09-11T01:20:32.315355+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:32.315355+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762112,"num_objects":7022,"num_object_clones":0,"num_object_copies":14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13216,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":1026}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13430","reported_seq":13506,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.496189+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:35.496189+0000","last_peered":"2024-09-11T01:20:35.496189+0000","last_clean":"2024-09-11T01:20:35.496189+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:35.496189+0000","last_undegraded":"2024-09-11T01:20:35.496189+0000","last_fullsized":"2024-09-11T01:20:35.496189+0000","mapping_epoch":31,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29049798,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11528,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13152","reported_seq":13294,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.556891+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:35.556891+0000","last_peered":"2024-09-11T01:20:35.556891+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:35.556891+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233752,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":877},{"shards":"2","objects":2920}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13343","reported_seq":13384,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.515606+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:35.515606+0000","last_peered":"2024-09-11T01:20:35.515606+0000","last_clean":"2024-09-11T01:20:35.515606+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:35.515606+0000","last_undegraded":"2024-09-11T01:20:35.515606+0000","last_fullsized":"2024-09-11T01:20:35.515606+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28749824,"num_objects":7019,"num_object_clones":0,"num_object_copies":14038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13228","reported_seq":13345,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.287820+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:32.287820+0000","last_peered":"2024-09-11T01:20:32.287820+0000","last_clean":"2024-09-11T01:20:32.287820+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:32.287820+0000","last_undegraded":"2024-09-11T01:20:32.287820+0000","last_fullsized":"2024-09-11T01:20:32.287820+0000","mapping_epoch":31,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28873187,"num_objects":7050,"num_object_clones":0,"num_object_copies":14100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7050,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10026,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13163","reported_seq":13394,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.305530+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:32.305530+0000","last_peered":"2024-09-11T01:20:32.305530+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:32.305530+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28852707,"num_objects":7045,"num_object_clones":0,"num_object_copies":14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7045,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9651,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2981},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13620","reported_seq":13863,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.272649+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:32.272649+0000","last_peered":"2024-09-11T01:20:32.272649+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:32.272649+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29459398,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10373,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":1072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13330","reported_seq":13445,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.240046+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:32.240046+0000","last_peered":"2024-09-11T01:20:32.240046+0000","last_clean":"2024-09-11T01:20:32.240046+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:32.240046+0000","last_undegraded":"2024-09-11T01:20:32.240046+0000","last_fullsized":"2024-09-11T01:20:32.240046+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13288","reported_seq":21627,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.767963+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:34.767963+0000","last_peered":"2024-09-11T01:20:34.767963+0000","last_clean":"2024-09-11T01:20:34.767963+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:34.767963+0000","last_undegraded":"2024-09-11T01:20:34.767963+0000","last_fullsized":"2024-09-11T01:20:34.767963+0000","mapping_epoch":40,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512739,"num_objects":6962,"num_object_clones":0,"num_object_copies":13924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11155,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13363","reported_seq":13606,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.214238+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:32.214238+0000","last_peered":"2024-09-11T01:20:32.214238+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:32.214238+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618176,"num_objects":7231,"num_object_clones":0,"num_object_copies":14462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9727,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3185},{"shards":"0,2","objects":1043}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13155","reported_seq":13323,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.563725+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:35.563725+0000","last_peered":"2024-09-11T01:20:35.563725+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:35.563725+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606947,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10924,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":944},{"shards":"2","objects":2997}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13503","reported_seq":13643,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.505301+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:35.505301+0000","last_peered":"2024-09-11T01:20:35.505301+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:35.505301+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11140,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":958},{"shards":"2","objects":2907}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13317","reported_seq":20840,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:32.290807+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:32.290807+0000","last_peered":"2024-09-11T01:20:32.290807+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:32.290807+0000","last_undegraded":"2024-09-11T01:20:32.290807+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28815360,"num_objects":7035,"num_object_clones":0,"num_object_copies":14070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":7035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3698,"num_bytes_recovered":15142912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6568},{"shards":"0,2","objects":467}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13432","reported_seq":13674,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.165967+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:32.165967+0000","last_peered":"2024-09-11T01:20:32.165967+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:32.165967+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29548544,"num_objects":7214,"num_object_clones":0,"num_object_copies":14428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9825,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3078},{"shards":"0,2","objects":1087}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13430","reported_seq":13695,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.167366+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:32.167366+0000","last_peered":"2024-09-11T01:20:32.167366+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:32.167366+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368320,"num_objects":7170,"num_object_clones":0,"num_object_copies":14340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9965,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3090},{"shards":"0,2","objects":1020}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13172","reported_seq":13386,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.293027+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:32.293027+0000","last_peered":"2024-09-11T01:20:32.293027+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:32.293027+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28533702,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9829,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2856},{"shards":"0,2","objects":1057}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":462955254,"num_objects":113036,"num_object_clones":0,"num_object_copies":226072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40853,"num_objects_misplaced":467,"num_objects_unfound":0,"num_objects_dirty":113036,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":172205,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8314,"num_bytes_recovered":32035299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1085788160,"data_stored":1085631883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160642,"ondisk_log_size":160642,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642312,"kb_used_data":357944,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93729528,"statfs":{"total":96636764160,"available":95979036672,"internally_reserved":0,"allocated":366534656,"data_stored":366263279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,1,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790016,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621868,"kb_used_data":297244,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93749972,"statfs":{"total":96636764160,"available":95999971328,"internally_reserved":0,"allocated":304377856,"data_stored":304062659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781820,"kb_used_data":407272,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93590020,"statfs":{"total":96636764160,"available":95836180480,"internally_reserved":0,"allocated":417046528,"data_stored":416775146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,0,0,0,3],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416313344,"data_stored":416277214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304001024,"data_stored":303921079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365473792,"data_stored":365433590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:41.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:42.047 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:42.048 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:42.070 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-11T01:20:40.553292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":456782601,"num_objects":111531,"num_object_clones":0,"num_object_copies":223062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40848,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":111531,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":173714,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8519,"num_bytes_recovered":32850422,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2034836,"kb_used_data":1055460,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":281080684,"statfs":{"total":289910292480,"available":287826620416,"internally_reserved":0,"allocated":1080791040,"data_stored":1079933084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1002818567},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,4,5,0,0,2,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18681856,"num_objects":-4561,"num_object_clones":0,"num_object_copies":-9122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-32,"num_objects_misplaced":-343,"num_objects_unfound":0,"num_objects_dirty":-4561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4561,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":1404928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004767"},"pg_stats":[{"pgid":"2.f","version":"41'13338","reported_seq":13531,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.206796+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:37.206796+0000","last_peered":"2024-09-11T01:20:37.206796+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:37.206796+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28262400,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13338,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2839},{"shards":"0,2","objects":904}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13430","reported_seq":13506,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.496189+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:35.496189+0000","last_peered":"2024-09-11T01:20:35.496189+0000","last_clean":"2024-09-11T01:20:35.496189+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:35.496189+0000","last_undegraded":"2024-09-11T01:20:35.496189+0000","last_fullsized":"2024-09-11T01:20:35.496189+0000","mapping_epoch":31,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29049798,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11528,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13152","reported_seq":13294,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.556891+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:35.556891+0000","last_peered":"2024-09-11T01:20:35.556891+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:35.556891+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28233752,"num_objects":6894,"num_object_clones":0,"num_object_copies":13788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6894,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11213,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":877},{"shards":"2","objects":2920}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13343","reported_seq":13384,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.515606+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:35.515606+0000","last_peered":"2024-09-11T01:20:35.515606+0000","last_clean":"2024-09-11T01:20:35.515606+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:35.515606+0000","last_undegraded":"2024-09-11T01:20:35.515606+0000","last_fullsized":"2024-09-11T01:20:35.515606+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28749824,"num_objects":7019,"num_object_clones":0,"num_object_copies":14038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13373","reported_seq":13490,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:37.318321+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:37.318321+0000","last_peered":"2024-09-11T01:20:37.318321+0000","last_clean":"2024-09-11T01:20:37.318321+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:37.318321+0000","last_undegraded":"2024-09-11T01:20:37.318321+0000","last_fullsized":"2024-09-11T01:20:37.318321+0000","mapping_epoch":31,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28279267,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10171,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13295","reported_seq":13527,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.296783+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:37.296783+0000","last_peered":"2024-09-11T01:20:37.296783+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:37.296783+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28312035,"num_objects":6913,"num_object_clones":0,"num_object_copies":13826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4065,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9783,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":51,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2849},{"shards":"0,2","objects":925}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13762","reported_seq":14011,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.316650+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:37.316650+0000","last_peered":"2024-09-11T01:20:37.316650+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:37.316650+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28877766,"num_objects":7052,"num_object_clones":0,"num_object_copies":14104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7052,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10515,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2829},{"shards":"0,2","objects":930}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13475","reported_seq":13590,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:37.302619+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:37.302619+0000","last_peered":"2024-09-11T01:20:37.302619+0000","last_clean":"2024-09-11T01:20:37.302619+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:37.302619+0000","last_undegraded":"2024-09-11T01:20:37.302619+0000","last_fullsized":"2024-09-11T01:20:37.302619+0000","mapping_epoch":31,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10428,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13458","reported_seq":21797,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:39.856425+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:39.856425+0000","last_peered":"2024-09-11T01:20:39.856425+0000","last_clean":"2024-09-11T01:20:39.856425+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:39.856425+0000","last_undegraded":"2024-09-11T01:20:39.856425+0000","last_fullsized":"2024-09-11T01:20:39.856425+0000","mapping_epoch":40,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816419,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11325,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893555+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:17:37.893555+0000","last_peered":"2024-09-11T01:17:37.893555+0000","last_clean":"2024-09-11T01:17:37.893555+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:17:37.893555+0000","last_undegraded":"2024-09-11T01:17:37.893555+0000","last_fullsized":"2024-09-11T01:17:37.893555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13501","reported_seq":13744,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.240394+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:37.240394+0000","last_peered":"2024-09-11T01:20:37.240394+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:37.240394+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29052928,"num_objects":7093,"num_object_clones":0,"num_object_copies":14186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9865,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3047},{"shards":"0,2","objects":905}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13155","reported_seq":13323,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.563725+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:35.563725+0000","last_peered":"2024-09-11T01:20:35.563725+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:35.563725+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3100","ondisk_log_start":"32'3100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28606947,"num_objects":6985,"num_object_clones":0,"num_object_copies":13970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10924,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":944},{"shards":"2","objects":2997}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13503","reported_seq":13643,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.505301+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:35.505301+0000","last_peered":"2024-09-11T01:20:35.505301+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:35.505301+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28913664,"num_objects":7059,"num_object_clones":0,"num_object_copies":14118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11140,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":958},{"shards":"2","objects":2907}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13455","reported_seq":21375,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:37.304110+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:37.304110+0000","last_peered":"2024-09-11T01:20:37.304110+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:37.304110+0000","last_undegraded":"2024-09-11T01:20:37.304110+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250112,"num_objects":6897,"num_object_clones":0,"num_object_copies":13794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":6897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10145,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15958016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6629},{"shards":"0,2","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13552","reported_seq":13798,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.331073+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:37.331073+0000","last_peered":"2024-09-11T01:20:37.331073+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:37.331073+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29057024,"num_objects":7094,"num_object_clones":0,"num_object_copies":14188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4137,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9945,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2960},{"shards":"0,2","objects":967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.894461+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:17:37.894461+0000","last_peered":"2024-09-11T01:17:37.894461+0000","last_clean":"2024-09-11T01:17:37.894461+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:17:37.894461+0000","last_undegraded":"2024-09-11T01:17:37.894461+0000","last_fullsized":"2024-09-11T01:17:37.894461+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13561","reported_seq":13826,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.285772+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:37.285772+0000","last_peered":"2024-09-11T01:20:37.285772+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:37.285772+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28831744,"num_objects":7039,"num_object_clones":0,"num_object_copies":14078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4082,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10096,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13296","reported_seq":13511,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.204805+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:37.204805+0000","last_peered":"2024-09-11T01:20:37.204805+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:37.204805+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28025798,"num_objects":6844,"num_object_clones":0,"num_object_copies":13688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9953,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2732},{"shards":"0,2","objects":933}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":456782582,"num_objects":111529,"num_object_clones":0,"num_object_copies":223058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40848,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":111529,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":173712,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8518,"num_bytes_recovered":32850403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1078722560,"data_stored":1078566283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642312,"kb_used_data":357944,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93729528,"statfs":{"total":96636764160,"available":95979036672,"internally_reserved":0,"allocated":366534656,"data_stored":366263279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":291172014},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,1,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613080,"kb_used_data":296776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93758760,"statfs":{"total":96636764160,"available":96008970240,"internally_reserved":0,"allocated":303898624,"data_stored":303583427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":779444,"kb_used_data":400740,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93592396,"statfs":{"total":96636764160,"available":95838613504,"internally_reserved":0,"allocated":410357760,"data_stored":410086378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,4,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":409722880,"data_stored":409686750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303525888,"data_stored":303445943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365473792,"data_stored":365433590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:42.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:42.645 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:42.646 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:42.677 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-11T01:20:42.553913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448262921,"num_objects":109451,"num_object_clones":0,"num_object_copies":218902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40844,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":109451,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":175794,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8718,"num_bytes_recovered":33649142,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161032,"ondisk_log_size":161032,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2036420,"kb_used_data":1044556,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281079100,"statfs":{"total":289910292480,"available":287824998400,"internally_reserved":0,"allocated":1069625344,"data_stored":1068767388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1015598087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,5,3,2,2,0,0,0,2,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26607616,"num_objects":-6496,"num_object_clones":0,"num_object_copies":-12992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-36,"num_objects_misplaced":-538,"num_objects_unfound":0,"num_objects_dirty":-6496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6496,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2203648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004603"},"pg_stats":[{"pgid":"2.f","version":"41'13497","reported_seq":13690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.335646+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:42.335646+0000","last_peered":"2024-09-11T01:20:42.335646+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:42.335646+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13497,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2680},{"shards":"0,2","objects":745}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13563","reported_seq":13639,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.528534+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:40.528534+0000","last_peered":"2024-09-11T01:20:40.528534+0000","last_clean":"2024-09-11T01:20:40.528534+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:40.528534+0000","last_undegraded":"2024-09-11T01:20:40.528534+0000","last_fullsized":"2024-09-11T01:20:40.528534+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28505030,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11661,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13280","reported_seq":13422,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:40.513601+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:40.513601+0000","last_peered":"2024-09-11T01:20:40.513601+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:40.513601+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27709464,"num_objects":6766,"num_object_clones":0,"num_object_copies":13532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11341,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":2792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13476","reported_seq":13517,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.508680+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:40.508680+0000","last_peered":"2024-09-11T01:20:40.508680+0000","last_clean":"2024-09-11T01:20:40.508680+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:40.508680+0000","last_undegraded":"2024-09-11T01:20:40.508680+0000","last_fullsized":"2024-09-11T01:20:40.508680+0000","mapping_epoch":31,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13507","reported_seq":13624,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.387860+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:42.387860+0000","last_peered":"2024-09-11T01:20:42.387860+0000","last_clean":"2024-09-11T01:20:42.387860+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:42.387860+0000","last_undegraded":"2024-09-11T01:20:42.387860+0000","last_fullsized":"2024-09-11T01:20:42.387860+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27730403,"num_objects":6771,"num_object_clones":0,"num_object_copies":13542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10305,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13450","reported_seq":13683,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.332941+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:42.332941+0000","last_peered":"2024-09-11T01:20:42.332941+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:42.332941+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27677155,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9938,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695},{"shards":"0,2","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13893","reported_seq":14144,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.358711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:42.358711+0000","last_peered":"2024-09-11T01:20:42.358711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:42.358711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28341190,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2699},{"shards":"0,2","objects":799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13599","reported_seq":13715,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.361630+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:42.361630+0000","last_peered":"2024-09-11T01:20:42.361630+0000","last_clean":"2024-09-11T01:20:42.361630+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:42.361630+0000","last_undegraded":"2024-09-11T01:20:42.361630+0000","last_fullsized":"2024-09-11T01:20:42.361630+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955200,"num_objects":6825,"num_object_clones":0,"num_object_copies":13650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13458","reported_seq":21797,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:39.856425+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:39.856425+0000","last_peered":"2024-09-11T01:20:39.856425+0000","last_clean":"2024-09-11T01:20:39.856425+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:39.856425+0000","last_undegraded":"2024-09-11T01:20:39.856425+0000","last_fullsized":"2024-09-11T01:20:39.856425+0000","mapping_epoch":40,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816419,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11325,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.280558+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:18:35.280558+0000","last_peered":"2024-09-11T01:18:35.280558+0000","last_clean":"2024-09-11T01:18:35.280558+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:18:35.280558+0000","last_undegraded":"2024-09-11T01:18:35.280558+0000","last_fullsized":"2024-09-11T01:18:35.280558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13643","reported_seq":13886,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.367784+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:42.367784+0000","last_peered":"2024-09-11T01:20:42.367784+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:42.367784+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":763}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13284","reported_seq":13452,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:40.592533+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:40.592533+0000","last_peered":"2024-09-11T01:20:40.592533+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:40.592533+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28078563,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11053,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":815},{"shards":"2","objects":2868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13670","reported_seq":13810,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:40.544169+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:40.544169+0000","last_peered":"2024-09-11T01:20:40.544169+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:40.544169+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28229632,"num_objects":6892,"num_object_clones":0,"num_object_copies":13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11307,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":791},{"shards":"2","objects":2740}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13585","reported_seq":21894,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:42.390194+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:42.390194+0000","last_peered":"2024-09-11T01:20:42.390194+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:42.390194+0000","last_undegraded":"2024-09-11T01:20:42.390194+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10275,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4092,"num_bytes_recovered":16756736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6694},{"shards":"0,2","objects":73}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13708","reported_seq":13956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.387182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:42.387182+0000","last_peered":"2024-09-11T01:20:42.387182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:42.387182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.282618+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:18:35.282618+0000","last_peered":"2024-09-11T01:18:35.282618+0000","last_clean":"2024-09-11T01:18:35.282618+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:18:35.282618+0000","last_undegraded":"2024-09-11T01:18:35.282618+0000","last_fullsized":"2024-09-11T01:18:35.282618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13690","reported_seq":13958,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.286748+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:42.286748+0000","last_peered":"2024-09-11T01:20:42.286748+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:42.286748+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28303360,"num_objects":6910,"num_object_clones":0,"num_object_copies":13820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2831},{"shards":"0,2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13426","reported_seq":13641,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.350143+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:42.350143+0000","last_peered":"2024-09-11T01:20:42.350143+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:42.350143+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27493318,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10083,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448262902,"num_objects":109449,"num_object_clones":0,"num_object_copies":218898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40844,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":109449,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":175792,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8717,"num_bytes_recovered":33649123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1067388928,"data_stored":1067232651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161029,"ondisk_log_size":161029,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":641952,"kb_used_data":353420,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93729888,"statfs":{"total":96636764160,"available":95979405312,"internally_reserved":0,"allocated":361902080,"data_stored":361630703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,1,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613080,"kb_used_data":296776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93758760,"statfs":{"total":96636764160,"available":96008970240,"internally_reserved":0,"allocated":303898624,"data_stored":303583427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781388,"kb_used_data":394360,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93590452,"statfs":{"total":96636764160,"available":95836622848,"internally_reserved":0,"allocated":403824640,"data_stored":403553258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":396291754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2,2,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402997248,"data_stored":402961118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303525888,"data_stored":303445943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360865792,"data_stored":360825590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:42.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:42.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'13497", 'reported_seq': 13690, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.335646+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:42.335646+0000', 'last_peered': '2024-09-11T01:20:42.335646+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:42.335646+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27611136, 'num_objects': 6741, 'num_object_clones': 0, 'num_object_copies': 13482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4061, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13497, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2680}, {'shards': '0,2', 'objects': 745}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:42.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "41'13280", 'reported_seq': 13422, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:40.513601+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:40.513601+0000', 'last_peered': '2024-09-11T01:20:40.513601+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:40.513601+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27709464, 'num_objects': 6766, 'num_object_clones': 0, 'num_object_copies': 13532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3975, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6766, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11341, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 32, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 749}, {'shards': '2', 'objects': 2792}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:42.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:42.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'13450", 'reported_seq': 13683, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.332941+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:42.332941+0000', 'last_peered': '2024-09-11T01:20:42.332941+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:42.332941+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27677155, 'num_objects': 6758, 'num_object_clones': 0, 'num_object_copies': 13516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4064, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6758, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9938, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2695}, {'shards': '0,2', 'objects': 770}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13893", 'reported_seq': 14144, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.358711+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:42.358711+0000', 'last_peered': '2024-09-11T01:20:42.358711+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:42.358711+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3800", 'ondisk_log_start': "32'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28341190, 'num_objects': 6921, 'num_object_clones': 0, 'num_object_copies': 13842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4222, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6921, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10646, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2699}, {'shards': '0,2', 'objects': 799}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'13643", 'reported_seq': 13886, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.367784+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:42.367784+0000', 'last_peered': '2024-09-11T01:20:42.367784+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:42.367784+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28471296, 'num_objects': 6951, 'num_object_clones': 0, 'num_object_copies': 13902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10007, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2905}, {'shards': '0,2', 'objects': 763}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "41'13284", 'reported_seq': 13452, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:40.592533+0000', 'last_change': '2024-09-11T01:15:20.515795+0000', 'last_active': '2024-09-11T01:20:40.592533+0000', 'last_peered': '2024-09-11T01:20:40.592533+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:40.592533+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'3200", 'ondisk_log_start': "32'3200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28078563, 'num_objects': 6856, 'num_object_clones': 0, 'num_object_copies': 13712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3989, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6856, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11053, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 815}, {'shards': '2', 'objects': 2868}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:42.680 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "41'13670", 'reported_seq': 13810, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:40.544169+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:40.544169+0000', 'last_peered': '2024-09-11T01:20:40.544169+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:40.544169+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28229632, 'num_objects': 6892, 'num_object_clones': 0, 'num_object_copies': 13784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6892, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11307, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 791}, {'shards': '2', 'objects': 2740}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'13585", 'reported_seq': 21894, 'reported_epoch': 41, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:20:42.390194+0000', 'last_change': '2024-09-11T01:17:36.615421+0000', 'last_active': '2024-09-11T01:20:42.390194+0000', 'last_peered': '2024-09-11T01:20:42.390194+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:42.390194+0000', 'last_undegraded': '2024-09-11T01:20:42.390194+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 38, 'log_start': "32'3500", 'ondisk_log_start': "32'3500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27717632, 'num_objects': 6767, 'num_object_clones': 0, 'num_object_copies': 13534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 73, 'num_objects_unfound': 0, 'num_objects_dirty': 6767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10275, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4092, 'num_bytes_recovered': 16756736, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6694}, {'shards': '0,2', 'objects': 73}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'13708", 'reported_seq': 13956, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.387182+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:42.387182+0000', 'last_peered': '2024-09-11T01:20:42.387182+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:42.387182+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28418048, 'num_objects': 6938, 'num_object_clones': 0, 'num_object_copies': 13876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4136, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2805}, {'shards': '0,2', 'objects': 811}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'13690", 'reported_seq': 13958, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.286748+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:42.286748+0000', 'last_peered': '2024-09-11T01:20:42.286748+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:42.286748+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28303360, 'num_objects': 6910, 'num_object_clones': 0, 'num_object_copies': 13820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4081, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6910, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10225, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2831}, {'shards': '0,2', 'objects': 760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:42.681 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'13426", 'reported_seq': 13641, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.350143+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:42.350143+0000', 'last_peered': '2024-09-11T01:20:42.350143+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:42.350143+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27493318, 'num_objects': 6714, 'num_object_clones': 0, 'num_object_copies': 13428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6714, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10083, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2602}, {'shards': '0,2', 'objects': 803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:42.681 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:43.411 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:43.411 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40844/218902 objects degraded (18.659%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":41,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109451,"data_bytes":448262921,"bytes_used":2085294080,"bytes_avail":287824998400,"bytes_total":289910292480,"degraded_objects":40844,"degraded_total":218902,"degraded_ratio":0.18658577811075275,"misplaced_objects":73,"misplaced_total":218902,"misplaced_ratio":0.00033348256297338537,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":183566,"recovering_keys_per_sec":0,"num_objects_recovered":574,"num_bytes_recovered":2203648,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":541},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:43.437 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:43.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:43.948 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:43.948 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:43.960 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-11T01:20:42.553913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":448262921,"num_objects":109451,"num_object_clones":0,"num_object_copies":218902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40844,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":109451,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":175794,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8718,"num_bytes_recovered":33649142,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161032,"ondisk_log_size":161032,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2036420,"kb_used_data":1044556,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":281079100,"statfs":{"total":289910292480,"available":287824998400,"internally_reserved":0,"allocated":1069625344,"data_stored":1068767388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1015598087},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,5,3,2,2,0,0,0,2,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26607616,"num_objects":-6496,"num_object_clones":0,"num_object_copies":-12992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-36,"num_objects_misplaced":-538,"num_objects_unfound":0,"num_objects_dirty":-6496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6496,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":574,"num_bytes_recovered":2203648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004603"},"pg_stats":[{"pgid":"2.f","version":"41'13497","reported_seq":13690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.335646+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:42.335646+0000","last_peered":"2024-09-11T01:20:42.335646+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:42.335646+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13497,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2680},{"shards":"0,2","objects":745}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"41'13563","reported_seq":13639,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.528534+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:40.528534+0000","last_peered":"2024-09-11T01:20:40.528534+0000","last_clean":"2024-09-11T01:20:40.528534+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:40.528534+0000","last_undegraded":"2024-09-11T01:20:40.528534+0000","last_fullsized":"2024-09-11T01:20:40.528534+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28505030,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11661,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"41'13280","reported_seq":13422,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:40.513601+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:40.513601+0000","last_peered":"2024-09-11T01:20:40.513601+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:40.513601+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27709464,"num_objects":6766,"num_object_clones":0,"num_object_copies":13532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11341,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":749},{"shards":"2","objects":2792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"41'13476","reported_seq":13517,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.508680+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:40.508680+0000","last_peered":"2024-09-11T01:20:40.508680+0000","last_clean":"2024-09-11T01:20:40.508680+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:40.508680+0000","last_undegraded":"2024-09-11T01:20:40.508680+0000","last_fullsized":"2024-09-11T01:20:40.508680+0000","mapping_epoch":31,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28205056,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13507","reported_seq":13624,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.387860+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:42.387860+0000","last_peered":"2024-09-11T01:20:42.387860+0000","last_clean":"2024-09-11T01:20:42.387860+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:42.387860+0000","last_undegraded":"2024-09-11T01:20:42.387860+0000","last_fullsized":"2024-09-11T01:20:42.387860+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27730403,"num_objects":6771,"num_object_clones":0,"num_object_copies":13542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10305,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13450","reported_seq":13683,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.332941+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:42.332941+0000","last_peered":"2024-09-11T01:20:42.332941+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:42.332941+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27677155,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9938,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695},{"shards":"0,2","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13893","reported_seq":14144,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.358711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:42.358711+0000","last_peered":"2024-09-11T01:20:42.358711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:42.358711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28341190,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2699},{"shards":"0,2","objects":799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13599","reported_seq":13715,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.361630+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:42.361630+0000","last_peered":"2024-09-11T01:20:42.361630+0000","last_clean":"2024-09-11T01:20:42.361630+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:42.361630+0000","last_undegraded":"2024-09-11T01:20:42.361630+0000","last_fullsized":"2024-09-11T01:20:42.361630+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955200,"num_objects":6825,"num_object_clones":0,"num_object_copies":13650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892089+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:17:37.892089+0000","last_peered":"2024-09-11T01:17:37.892089+0000","last_clean":"2024-09-11T01:17:37.892089+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:17:37.892089+0000","last_undegraded":"2024-09-11T01:17:37.892089+0000","last_fullsized":"2024-09-11T01:17:37.892089+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"41'13458","reported_seq":21797,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:39.856425+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:39.856425+0000","last_peered":"2024-09-11T01:20:39.856425+0000","last_clean":"2024-09-11T01:20:39.856425+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:39.856425+0000","last_undegraded":"2024-09-11T01:20:39.856425+0000","last_fullsized":"2024-09-11T01:20:39.856425+0000","mapping_epoch":40,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27816419,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11325,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.280558+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:18:35.280558+0000","last_peered":"2024-09-11T01:18:35.280558+0000","last_clean":"2024-09-11T01:18:35.280558+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:18:35.280558+0000","last_undegraded":"2024-09-11T01:18:35.280558+0000","last_fullsized":"2024-09-11T01:18:35.280558+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13643","reported_seq":13886,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.367784+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:42.367784+0000","last_peered":"2024-09-11T01:20:42.367784+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:42.367784+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":763}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"41'13284","reported_seq":13452,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:40.592533+0000","last_change":"2024-09-11T01:15:20.515795+0000","last_active":"2024-09-11T01:20:40.592533+0000","last_peered":"2024-09-11T01:20:40.592533+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:40.592533+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3200","ondisk_log_start":"32'3200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28078563,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11053,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":815},{"shards":"2","objects":2868}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892427+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:17:37.892427+0000","last_peered":"2024-09-11T01:17:37.892427+0000","last_clean":"2024-09-11T01:17:37.892427+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:17:37.892427+0000","last_undegraded":"2024-09-11T01:17:37.892427+0000","last_fullsized":"2024-09-11T01:17:37.892427+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"41'13670","reported_seq":13810,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:40.544169+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:40.544169+0000","last_peered":"2024-09-11T01:20:40.544169+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:40.544169+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28229632,"num_objects":6892,"num_object_clones":0,"num_object_copies":13784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11307,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":791},{"shards":"2","objects":2740}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13585","reported_seq":21894,"reported_epoch":41,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:20:42.390194+0000","last_change":"2024-09-11T01:17:36.615421+0000","last_active":"2024-09-11T01:20:42.390194+0000","last_peered":"2024-09-11T01:20:42.390194+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:42.390194+0000","last_undegraded":"2024-09-11T01:20:42.390194+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10275,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4092,"num_bytes_recovered":16756736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6694},{"shards":"0,2","objects":73}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892550+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:17:37.892550+0000","last_peered":"2024-09-11T01:17:37.892550+0000","last_clean":"2024-09-11T01:17:37.892550+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:17:37.892550+0000","last_undegraded":"2024-09-11T01:17:37.892550+0000","last_fullsized":"2024-09-11T01:17:37.892550+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13708","reported_seq":13956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.387182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:42.387182+0000","last_peered":"2024-09-11T01:20:42.387182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:42.387182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":57,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.282618+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:18:35.282618+0000","last_peered":"2024-09-11T01:18:35.282618+0000","last_clean":"2024-09-11T01:18:35.282618+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:18:35.282618+0000","last_undegraded":"2024-09-11T01:18:35.282618+0000","last_fullsized":"2024-09-11T01:18:35.282618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13690","reported_seq":13958,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.286748+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:42.286748+0000","last_peered":"2024-09-11T01:20:42.286748+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:42.286748+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28303360,"num_objects":6910,"num_object_clones":0,"num_object_copies":13820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2831},{"shards":"0,2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.892315+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:17:37.892315+0000","last_peered":"2024-09-11T01:17:37.892315+0000","last_clean":"2024-09-11T01:17:37.892315+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:17:37.892315+0000","last_undegraded":"2024-09-11T01:17:37.892315+0000","last_fullsized":"2024-09-11T01:17:37.892315+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:17:37.893954+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:17:37.893954+0000","last_peered":"2024-09-11T01:17:37.893954+0000","last_clean":"2024-09-11T01:17:37.893954+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:17:37.893954+0000","last_undegraded":"2024-09-11T01:17:37.893954+0000","last_fullsized":"2024-09-11T01:17:37.893954+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13426","reported_seq":13641,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.350143+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:42.350143+0000","last_peered":"2024-09-11T01:20:42.350143+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:42.350143+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27493318,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10083,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":448262902,"num_objects":109449,"num_object_clones":0,"num_object_copies":218898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40844,"num_objects_misplaced":73,"num_objects_unfound":0,"num_objects_dirty":109449,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":175792,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8717,"num_bytes_recovered":33649123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1067388928,"data_stored":1067232651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161029,"ondisk_log_size":161029,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":641952,"kb_used_data":353420,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93729888,"statfs":{"total":96636764160,"available":95979405312,"internally_reserved":0,"allocated":361902080,"data_stored":361630703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,3,1,0,0,0,0,0,0,5],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790017,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613080,"kb_used_data":296776,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93758760,"statfs":{"total":96636764160,"available":96008970240,"internally_reserved":0,"allocated":303898624,"data_stored":303583427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781388,"kb_used_data":394360,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93590452,"statfs":{"total":96636764160,"available":95836622848,"internally_reserved":0,"allocated":403824640,"data_stored":403553258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":396291754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2,2,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402997248,"data_stored":402961118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303525888,"data_stored":303445943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360865792,"data_stored":360825590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:46.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:47.528 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:47.530 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:47.549 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-11T01:20:46.555134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":445260553,"num_objects":108718,"num_object_clones":0,"num_object_copies":217436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40115,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":108718,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":176527,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8879,"num_bytes_recovered":34280409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2054408,"kb_used_data":1040332,"kb_used_omap":12,"kb_used_meta":1014003,"kb_avail":281061112,"statfs":{"total":289910292480,"available":287806578688,"internally_reserved":0,"allocated":1065299968,"data_stored":1064440410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1038339079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,4,7,5,2,0,0,0,2,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12218368,"num_objects":-2983,"num_object_clones":0,"num_object_copies":-5966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-733,"num_objects_misplaced":373,"num_objects_unfound":0,"num_objects_dirty":-2983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2983,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":1429987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002500"},"pg_stats":[{"pgid":"2.f","version":"41'13497","reported_seq":13690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.335646+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:42.335646+0000","last_peered":"2024-09-11T01:20:42.335646+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:42.335646+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13497,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2680},{"shards":"0,2","objects":745}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'13663","reported_seq":13741,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.136694+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:45.136694+0000","last_peered":"2024-09-11T01:20:45.136694+0000","last_clean":"2024-09-11T01:20:45.136694+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:45.136694+0000","last_undegraded":"2024-09-11T01:20:45.136694+0000","last_fullsized":"2024-09-11T01:20:45.136694+0000","mapping_epoch":31,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28095430,"num_objects":6861,"num_object_clones":0,"num_object_copies":13722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6861,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11761,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"42'13386","reported_seq":13538,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:44.972191+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:44.972191+0000","last_peered":"2024-09-11T01:20:44.972191+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:44.972191+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27275288,"num_objects":6660,"num_object_clones":0,"num_object_copies":13320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11447,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":643},{"shards":"2","objects":2690}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'13587","reported_seq":13630,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.131897+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:45.131897+0000","last_peered":"2024-09-11T01:20:45.131897+0000","last_clean":"2024-09-11T01:20:45.131897+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:45.131897+0000","last_undegraded":"2024-09-11T01:20:45.131897+0000","last_fullsized":"2024-09-11T01:20:45.131897+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13587,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13507","reported_seq":13624,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.387860+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:42.387860+0000","last_peered":"2024-09-11T01:20:42.387860+0000","last_clean":"2024-09-11T01:20:42.387860+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:42.387860+0000","last_undegraded":"2024-09-11T01:20:42.387860+0000","last_fullsized":"2024-09-11T01:20:42.387860+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27730403,"num_objects":6771,"num_object_clones":0,"num_object_copies":13542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10305,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13450","reported_seq":13683,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.332941+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:42.332941+0000","last_peered":"2024-09-11T01:20:42.332941+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:42.332941+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27677155,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9938,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695},{"shards":"0,2","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13893","reported_seq":14144,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.358711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:42.358711+0000","last_peered":"2024-09-11T01:20:42.358711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:42.358711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28341190,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2699},{"shards":"0,2","objects":799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13599","reported_seq":13715,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.361630+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:42.361630+0000","last_peered":"2024-09-11T01:20:42.361630+0000","last_clean":"2024-09-11T01:20:42.361630+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:42.361630+0000","last_undegraded":"2024-09-11T01:20:42.361630+0000","last_fullsized":"2024-09-11T01:20:42.361630+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955200,"num_objects":6825,"num_object_clones":0,"num_object_copies":13650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.631496+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:44.631496+0000","last_peered":"2024-09-11T01:20:44.631496+0000","last_clean":"2024-09-11T01:20:44.631496+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:44.631496+0000","last_undegraded":"2024-09-11T01:20:44.631496+0000","last_fullsized":"2024-09-11T01:20:44.631496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"42'13592","reported_seq":21933,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.879772+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:44.879772+0000","last_peered":"2024-09-11T01:20:44.879772+0000","last_clean":"2024-09-11T01:20:44.879772+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:44.879772+0000","last_undegraded":"2024-09-11T01:20:44.879772+0000","last_fullsized":"2024-09-11T01:20:44.879772+0000","mapping_epoch":40,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267555,"num_objects":6658,"num_object_clones":0,"num_object_copies":13316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11459,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":59,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.619172+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:44.619172+0000","last_peered":"2024-09-11T01:20:44.619172+0000","last_clean":"2024-09-11T01:20:44.619172+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:44.619172+0000","last_undegraded":"2024-09-11T01:20:44.619172+0000","last_fullsized":"2024-09-11T01:20:44.619172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13643","reported_seq":13886,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.367784+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:42.367784+0000","last_peered":"2024-09-11T01:20:42.367784+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:42.367784+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":763}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"42'13403","reported_seq":13743,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.612089+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:45.612089+0000","last_peered":"2024-09-11T01:20:45.612089+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:45.612089+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591139,"num_objects":6737,"num_object_clones":0,"num_object_copies":13474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3264,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":6737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11172,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":332259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":641},{"shards":"2","objects":2832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":82,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.614736+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:44.614736+0000","last_peered":"2024-09-11T01:20:44.614736+0000","last_clean":"2024-09-11T01:20:44.614736+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:44.614736+0000","last_undegraded":"2024-09-11T01:20:44.614736+0000","last_fullsized":"2024-09-11T01:20:44.614736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"42'13786","reported_seq":13928,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.088736+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:45.088736+0000","last_peered":"2024-09-11T01:20:45.088736+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:45.088736+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11423,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":675},{"shards":"2","objects":2624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13632","reported_seq":22094,"reported_epoch":42,"state":"peering","last_fresh":"2024-09-11T01:20:44.589340+0000","last_change":"2024-09-11T01:20:44.933985+0000","last_active":"2024-09-11T01:20:44.933985+0000","last_peered":"2024-09-11T01:20:44.589340+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:44.589340+0000","last_undegraded":"2024-09-11T01:20:44.589340+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":42,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.630994+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:44.630994+0000","last_peered":"2024-09-11T01:20:44.630994+0000","last_clean":"2024-09-11T01:20:44.630994+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:44.630994+0000","last_undegraded":"2024-09-11T01:20:44.630994+0000","last_fullsized":"2024-09-11T01:20:44.630994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13708","reported_seq":13956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.387182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:42.387182+0000","last_peered":"2024-09-11T01:20:42.387182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:42.387182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.618683+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:44.618683+0000","last_peered":"2024-09-11T01:20:44.618683+0000","last_clean":"2024-09-11T01:20:44.618683+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:44.618683+0000","last_undegraded":"2024-09-11T01:20:44.618683+0000","last_fullsized":"2024-09-11T01:20:44.618683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13690","reported_seq":13958,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.286748+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:42.286748+0000","last_peered":"2024-09-11T01:20:42.286748+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:42.286748+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28303360,"num_objects":6910,"num_object_clones":0,"num_object_copies":13820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2831},{"shards":"0,2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.615159+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:44.615159+0000","last_peered":"2024-09-11T01:20:44.615159+0000","last_clean":"2024-09-11T01:20:44.615159+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:44.615159+0000","last_undegraded":"2024-09-11T01:20:44.615159+0000","last_fullsized":"2024-09-11T01:20:44.615159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.651606+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:44.651606+0000","last_peered":"2024-09-11T01:20:44.651606+0000","last_clean":"2024-09-11T01:20:44.651606+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:44.651606+0000","last_undegraded":"2024-09-11T01:20:44.651606+0000","last_fullsized":"2024-09-11T01:20:44.651606+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13426","reported_seq":13641,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.350143+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:42.350143+0000","last_peered":"2024-09-11T01:20:42.350143+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:42.350143+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27493318,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10083,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":445260534,"num_objects":108716,"num_object_clones":0,"num_object_copies":217432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40115,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":108716,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":176525,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8878,"num_bytes_recovered":34280390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1063522304,"data_stored":1063369640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160962,"ondisk_log_size":160962,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637932,"kb_used_data":349396,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93733908,"statfs":{"total":96636764160,"available":95983521792,"internally_reserved":0,"allocated":357781504,"data_stored":357504912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,2,3,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635088,"kb_used_data":296576,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93736752,"statfs":{"total":96636764160,"available":95986434048,"internally_reserved":0,"allocated":303693824,"data_stored":303382240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781388,"kb_used_data":394360,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93590452,"statfs":{"total":96636764160,"available":95836622848,"internally_reserved":0,"allocated":403824640,"data_stored":403553258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":396291754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2,2,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402997248,"data_stored":402961118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303288320,"data_stored":303211988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":357236736,"data_stored":357196534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:47.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:48.123 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:48.123 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:48.155 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-11T01:20:46.555134+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":445260553,"num_objects":108718,"num_object_clones":0,"num_object_copies":217436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40115,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":108718,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":176527,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8879,"num_bytes_recovered":34280409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2054408,"kb_used_data":1040332,"kb_used_omap":12,"kb_used_meta":1014003,"kb_avail":281061112,"statfs":{"total":289910292480,"available":287806578688,"internally_reserved":0,"allocated":1065299968,"data_stored":1064440410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1038339079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,4,7,5,2,0,0,0,2,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12218368,"num_objects":-2983,"num_object_clones":0,"num_object_copies":-5966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-733,"num_objects_misplaced":373,"num_objects_unfound":0,"num_objects_dirty":-2983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2983,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":360,"num_bytes_recovered":1429987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002500"},"pg_stats":[{"pgid":"2.f","version":"41'13497","reported_seq":13690,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.335646+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:42.335646+0000","last_peered":"2024-09-11T01:20:42.335646+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:42.335646+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27611136,"num_objects":6741,"num_object_clones":0,"num_object_copies":13482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6741,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13497,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2680},{"shards":"0,2","objects":745}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'13663","reported_seq":13741,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.136694+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:45.136694+0000","last_peered":"2024-09-11T01:20:45.136694+0000","last_clean":"2024-09-11T01:20:45.136694+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:45.136694+0000","last_undegraded":"2024-09-11T01:20:45.136694+0000","last_fullsized":"2024-09-11T01:20:45.136694+0000","mapping_epoch":31,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28095430,"num_objects":6861,"num_object_clones":0,"num_object_copies":13722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6861,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11761,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"42'13386","reported_seq":13538,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:44.972191+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:44.972191+0000","last_peered":"2024-09-11T01:20:44.972191+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:44.972191+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27275288,"num_objects":6660,"num_object_clones":0,"num_object_copies":13320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11447,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":643},{"shards":"2","objects":2690}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'13587","reported_seq":13630,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.131897+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:45.131897+0000","last_peered":"2024-09-11T01:20:45.131897+0000","last_clean":"2024-09-11T01:20:45.131897+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:45.131897+0000","last_undegraded":"2024-09-11T01:20:45.131897+0000","last_fullsized":"2024-09-11T01:20:45.131897+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13587,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"41'13507","reported_seq":13624,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.387860+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:42.387860+0000","last_peered":"2024-09-11T01:20:42.387860+0000","last_clean":"2024-09-11T01:20:42.387860+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:42.387860+0000","last_undegraded":"2024-09-11T01:20:42.387860+0000","last_fullsized":"2024-09-11T01:20:42.387860+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27730403,"num_objects":6771,"num_object_clones":0,"num_object_copies":13542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10305,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"41'13450","reported_seq":13683,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.332941+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:42.332941+0000","last_peered":"2024-09-11T01:20:42.332941+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:42.332941+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27677155,"num_objects":6758,"num_object_clones":0,"num_object_copies":13516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6758,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9938,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2695},{"shards":"0,2","objects":770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"41'13893","reported_seq":14144,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.358711+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:42.358711+0000","last_peered":"2024-09-11T01:20:42.358711+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:42.358711+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28341190,"num_objects":6921,"num_object_clones":0,"num_object_copies":13842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4222,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6921,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10646,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2699},{"shards":"0,2","objects":799}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"41'13599","reported_seq":13715,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.361630+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:42.361630+0000","last_peered":"2024-09-11T01:20:42.361630+0000","last_clean":"2024-09-11T01:20:42.361630+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:42.361630+0000","last_undegraded":"2024-09-11T01:20:42.361630+0000","last_fullsized":"2024-09-11T01:20:42.361630+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955200,"num_objects":6825,"num_object_clones":0,"num_object_copies":13650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10552,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.631496+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:44.631496+0000","last_peered":"2024-09-11T01:20:44.631496+0000","last_clean":"2024-09-11T01:20:44.631496+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:44.631496+0000","last_undegraded":"2024-09-11T01:20:44.631496+0000","last_fullsized":"2024-09-11T01:20:44.631496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"42'13592","reported_seq":21933,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.879772+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:44.879772+0000","last_peered":"2024-09-11T01:20:44.879772+0000","last_clean":"2024-09-11T01:20:44.879772+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:44.879772+0000","last_undegraded":"2024-09-11T01:20:44.879772+0000","last_fullsized":"2024-09-11T01:20:44.879772+0000","mapping_epoch":40,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267555,"num_objects":6658,"num_object_clones":0,"num_object_copies":13316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11459,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":59,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.619172+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:44.619172+0000","last_peered":"2024-09-11T01:20:44.619172+0000","last_clean":"2024-09-11T01:20:44.619172+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:44.619172+0000","last_undegraded":"2024-09-11T01:20:44.619172+0000","last_fullsized":"2024-09-11T01:20:44.619172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"41'13643","reported_seq":13886,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.367784+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:42.367784+0000","last_peered":"2024-09-11T01:20:42.367784+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:42.367784+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10007,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2905},{"shards":"0,2","objects":763}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":94,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.217327+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:18:21.217327+0000","last_peered":"2024-09-11T01:18:21.217327+0000","last_clean":"2024-09-11T01:18:21.217327+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:18:21.217327+0000","last_undegraded":"2024-09-11T01:18:21.217327+0000","last_fullsized":"2024-09-11T01:18:21.217327+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"42'13403","reported_seq":13743,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.612089+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:45.612089+0000","last_peered":"2024-09-11T01:20:45.612089+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:45.612089+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591139,"num_objects":6737,"num_object_clones":0,"num_object_copies":13474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3264,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":6737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11172,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":332259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":641},{"shards":"2","objects":2832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":82,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.614736+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:44.614736+0000","last_peered":"2024-09-11T01:20:44.614736+0000","last_clean":"2024-09-11T01:20:44.614736+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:44.614736+0000","last_undegraded":"2024-09-11T01:20:44.614736+0000","last_fullsized":"2024-09-11T01:20:44.614736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"42'13786","reported_seq":13928,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.088736+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:45.088736+0000","last_peered":"2024-09-11T01:20:45.088736+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:45.088736+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11423,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":675},{"shards":"2","objects":2624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13632","reported_seq":22094,"reported_epoch":42,"state":"peering","last_fresh":"2024-09-11T01:20:44.589340+0000","last_change":"2024-09-11T01:20:44.933985+0000","last_active":"2024-09-11T01:20:44.933985+0000","last_peered":"2024-09-11T01:20:44.589340+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:44.589340+0000","last_undegraded":"2024-09-11T01:20:44.589340+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":42,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.630994+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:44.630994+0000","last_peered":"2024-09-11T01:20:44.630994+0000","last_clean":"2024-09-11T01:20:44.630994+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:44.630994+0000","last_undegraded":"2024-09-11T01:20:44.630994+0000","last_fullsized":"2024-09-11T01:20:44.630994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"41'13708","reported_seq":13956,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.387182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:42.387182+0000","last_peered":"2024-09-11T01:20:42.387182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:42.387182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28418048,"num_objects":6938,"num_object_clones":0,"num_object_copies":13876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":811}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.618683+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:44.618683+0000","last_peered":"2024-09-11T01:20:44.618683+0000","last_clean":"2024-09-11T01:20:44.618683+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:44.618683+0000","last_undegraded":"2024-09-11T01:20:44.618683+0000","last_fullsized":"2024-09-11T01:20:44.618683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"41'13690","reported_seq":13958,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.286748+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:42.286748+0000","last_peered":"2024-09-11T01:20:42.286748+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:42.286748+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28303360,"num_objects":6910,"num_object_clones":0,"num_object_copies":13820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4081,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":69,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2831},{"shards":"0,2","objects":760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.615159+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:44.615159+0000","last_peered":"2024-09-11T01:20:44.615159+0000","last_clean":"2024-09-11T01:20:44.615159+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:44.615159+0000","last_undegraded":"2024-09-11T01:20:44.615159+0000","last_fullsized":"2024-09-11T01:20:44.615159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.651606+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:44.651606+0000","last_peered":"2024-09-11T01:20:44.651606+0000","last_clean":"2024-09-11T01:20:44.651606+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:44.651606+0000","last_undegraded":"2024-09-11T01:20:44.651606+0000","last_fullsized":"2024-09-11T01:20:44.651606+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"41'13426","reported_seq":13641,"reported_epoch":41,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.350143+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:42.350143+0000","last_peered":"2024-09-11T01:20:42.350143+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:42.350143+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27493318,"num_objects":6714,"num_object_clones":0,"num_object_copies":13428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4114,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10083,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2602},{"shards":"0,2","objects":803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":445260534,"num_objects":108716,"num_object_clones":0,"num_object_copies":217432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40115,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":108716,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":176525,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8878,"num_bytes_recovered":34280390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1063522304,"data_stored":1063369640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160962,"ondisk_log_size":160962,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637932,"kb_used_data":349396,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93733908,"statfs":{"total":96636764160,"available":95983521792,"internally_reserved":0,"allocated":357781504,"data_stored":357504912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,2,3,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635088,"kb_used_data":296576,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93736752,"statfs":{"total":96636764160,"available":95986434048,"internally_reserved":0,"allocated":303693824,"data_stored":303382240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781388,"kb_used_data":394360,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93590452,"statfs":{"total":96636764160,"available":95836622848,"internally_reserved":0,"allocated":403824640,"data_stored":403553258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":396291754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2,2,2,0,0,0,2],"upper_bound":4096},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":402997248,"data_stored":402961118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303288320,"data_stored":303211988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":357236736,"data_stored":357196534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:48.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:48.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "41'13497", 'reported_seq': 13690, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.335646+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:42.335646+0000', 'last_peered': '2024-09-11T01:20:42.335646+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:42.335646+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27611136, 'num_objects': 6741, 'num_object_clones': 0, 'num_object_copies': 13482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4061, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6741, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13497, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2680}, {'shards': '0,2', 'objects': 745}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "42'13386", 'reported_seq': 13538, 'reported_epoch': 42, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:44.972191+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:44.972191+0000', 'last_peered': '2024-09-11T01:20:44.972191+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:44.972191+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'3300", 'ondisk_log_start': "32'3300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27275288, 'num_objects': 6660, 'num_object_clones': 0, 'num_object_copies': 13320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3971, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6660, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11447, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 36, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 643}, {'shards': '2', 'objects': 2690}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "41'13450", 'reported_seq': 13683, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.332941+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:42.332941+0000', 'last_peered': '2024-09-11T01:20:42.332941+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:42.332941+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27677155, 'num_objects': 6758, 'num_object_clones': 0, 'num_object_copies': 13516, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4064, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6758, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9938, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2695}, {'shards': '0,2', 'objects': 770}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "41'13893", 'reported_seq': 14144, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.358711+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:42.358711+0000', 'last_peered': '2024-09-11T01:20:42.358711+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:42.358711+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "32'3800", 'ondisk_log_start': "32'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28341190, 'num_objects': 6921, 'num_object_clones': 0, 'num_object_copies': 13842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4222, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6921, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10646, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2699}, {'shards': '0,2', 'objects': 799}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:48.157 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "41'13643", 'reported_seq': 13886, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.367784+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:42.367784+0000', 'last_peered': '2024-09-11T01:20:42.367784+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:42.367784+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28471296, 'num_objects': 6951, 'num_object_clones': 0, 'num_object_copies': 13902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10007, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2905}, {'shards': '0,2', 'objects': 763}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "42'13403", 'reported_seq': 13743, 'reported_epoch': 42, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:45.612089+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:20:45.612089+0000', 'last_peered': '2024-09-11T01:20:45.612089+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:45.612089+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27591139, 'num_objects': 6737, 'num_object_clones': 0, 'num_object_copies': 13474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3264, 'num_objects_misplaced': 641, 'num_objects_unfound': 0, 'num_objects_dirty': 6737, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11172, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 123, 'num_bytes_recovered': 332259, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 641}, {'shards': '2', 'objects': 2832}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "42'13786", 'reported_seq': 13928, 'reported_epoch': 42, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:45.088736+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:45.088736+0000', 'last_peered': '2024-09-11T01:20:45.088736+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:45.088736+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27754496, 'num_objects': 6776, 'num_object_clones': 0, 'num_object_copies': 13552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4154, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11423, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 25, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 675}, {'shards': '2', 'objects': 2624}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "41'13632", 'reported_seq': 22094, 'reported_epoch': 42, 'state': 'peering', 'last_fresh': '2024-09-11T01:20:44.589340+0000', 'last_change': '2024-09-11T01:20:44.933985+0000', 'last_active': '2024-09-11T01:20:44.933985+0000', 'last_peered': '2024-09-11T01:20:44.589340+0000', 'last_clean': '2024-09-11T01:15:17.959258+0000', 'last_became_active': '2024-09-11T01:15:20.470479+0000', 'last_became_peered': '2024-09-11T01:15:20.470479+0000', 'last_unstale': '2024-09-11T01:20:44.589340+0000', 'last_undegraded': '2024-09-11T01:20:44.589340+0000', 'last_fullsized': '2024-09-11T01:15:19.985051+0000', 'mapping_epoch': 42, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3367", 'last_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.048814+0000', 'objects_scrubbed': 3308, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1209, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27525120, 'num_objects': 6720, 'num_object_clones': 0, 'num_object_copies': 13440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10322, 'num_write_kb': 27468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4165, 'num_bytes_recovered': 17055744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "41'13708", 'reported_seq': 13956, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.387182+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:42.387182+0000', 'last_peered': '2024-09-11T01:20:42.387182+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:42.387182+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28418048, 'num_objects': 6938, 'num_object_clones': 0, 'num_object_copies': 13876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4136, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 59, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2805}, {'shards': '0,2', 'objects': 811}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:48.158 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "41'13690", 'reported_seq': 13958, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.286748+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:42.286748+0000', 'last_peered': '2024-09-11T01:20:42.286748+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:42.286748+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28303360, 'num_objects': 6910, 'num_object_clones': 0, 'num_object_copies': 13820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4081, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6910, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10225, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 69, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2831}, {'shards': '0,2', 'objects': 760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.159 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:48.159 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "41'13426", 'reported_seq': 13641, 'reported_epoch': 41, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.350143+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:42.350143+0000', 'last_peered': '2024-09-11T01:20:42.350143+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:42.350143+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'3400", 'ondisk_log_start': "32'3400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27493318, 'num_objects': 6714, 'num_object_clones': 0, 'num_object_copies': 13428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4114, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6714, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10083, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 42, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2602}, {'shards': '0,2', 'objects': 803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:48.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:48.840 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:48.841 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40091/215390 objects degraded (18.613%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":107695,"data_bytes":441070345,"bytes_used":2085457920,"bytes_avail":287824834560,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":40091,"degraded_total":215390,"degraded_ratio":0.18613213241097543,"misplaced_objects":641,"misplaced_total":215390,"misplaced_ratio":0.0029759970286457123,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":178688,"recovering_keys_per_sec":0,"num_objects_recovered":384,"num_bytes_recovered":1429987,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":479},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:48.864 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:48.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:49.433 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:49.433 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:49.466 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-11T01:20:48.555987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":441070345,"num_objects":107695,"num_object_clones":0,"num_object_copies":215390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40091,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":107695,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":177550,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8903,"num_bytes_recovered":34280409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2036580,"kb_used_data":1034976,"kb_used_omap":12,"kb_used_meta":1001523,"kb_avail":281078940,"statfs":{"total":289910292480,"available":287824834560,"internally_reserved":0,"allocated":1059815424,"data_stored":1058945415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1025559559},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,2,5,4,7,3,0,0,0,8,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15712256,"num_objects":-3836,"num_object_clones":0,"num_object_copies":-7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-757,"num_objects_misplaced":373,"num_objects_unfound":0,"num_objects_dirty":-3836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3836,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":1429987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002695"},"pg_stats":[{"pgid":"2.f","version":"43'13586","reported_seq":13787,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.231927+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:47.231927+0000","last_peered":"2024-09-11T01:20:47.231927+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:47.231927+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27246592,"num_objects":6652,"num_object_clones":0,"num_object_copies":13304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13586,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2593},{"shards":"0,2","objects":656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"42'13663","reported_seq":13741,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.136694+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:45.136694+0000","last_peered":"2024-09-11T01:20:45.136694+0000","last_clean":"2024-09-11T01:20:45.136694+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:45.136694+0000","last_undegraded":"2024-09-11T01:20:45.136694+0000","last_fullsized":"2024-09-11T01:20:45.136694+0000","mapping_epoch":31,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28095430,"num_objects":6861,"num_object_clones":0,"num_object_copies":13722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6861,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11761,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"42'13386","reported_seq":13538,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:44.972191+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:44.972191+0000","last_peered":"2024-09-11T01:20:44.972191+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:44.972191+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3300","ondisk_log_start":"32'3300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27275288,"num_objects":6660,"num_object_clones":0,"num_object_copies":13320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11447,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":643},{"shards":"2","objects":2690}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"42'13587","reported_seq":13630,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.131897+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:45.131897+0000","last_peered":"2024-09-11T01:20:45.131897+0000","last_clean":"2024-09-11T01:20:45.131897+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:45.131897+0000","last_undegraded":"2024-09-11T01:20:45.131897+0000","last_fullsized":"2024-09-11T01:20:45.131897+0000","mapping_epoch":31,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13587,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13607","reported_seq":13729,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.412436+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:47.412436+0000","last_peered":"2024-09-11T01:20:47.412436+0000","last_clean":"2024-09-11T01:20:47.412436+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:47.412436+0000","last_undegraded":"2024-09-11T01:20:47.412436+0000","last_fullsized":"2024-09-11T01:20:47.412436+0000","mapping_epoch":31,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27320803,"num_objects":6671,"num_object_clones":0,"num_object_copies":13342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6671,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10405,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13582","reported_seq":13828,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.353368+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:47.353368+0000","last_peered":"2024-09-11T01:20:47.353368+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:47.353368+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27136483,"num_objects":6626,"num_object_clones":0,"num_object_copies":13252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4060,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10070,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":56,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2567},{"shards":"0,2","objects":638}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14012","reported_seq":14275,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.306275+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:47.306275+0000","last_peered":"2024-09-11T01:20:47.306275+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:47.306275+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27853766,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10765,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2584},{"shards":"0,2","objects":680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13705","reported_seq":13825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.224901+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:47.224901+0000","last_peered":"2024-09-11T01:20:47.224901+0000","last_clean":"2024-09-11T01:20:47.224901+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:47.224901+0000","last_undegraded":"2024-09-11T01:20:47.224901+0000","last_fullsized":"2024-09-11T01:20:47.224901+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27521024,"num_objects":6719,"num_object_clones":0,"num_object_copies":13438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10658,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.631496+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:44.631496+0000","last_peered":"2024-09-11T01:20:44.631496+0000","last_clean":"2024-09-11T01:20:44.631496+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:44.631496+0000","last_undegraded":"2024-09-11T01:20:44.631496+0000","last_fullsized":"2024-09-11T01:20:44.631496+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"42'13592","reported_seq":21933,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.879772+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:44.879772+0000","last_peered":"2024-09-11T01:20:44.879772+0000","last_clean":"2024-09-11T01:20:44.879772+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:44.879772+0000","last_undegraded":"2024-09-11T01:20:44.879772+0000","last_fullsized":"2024-09-11T01:20:44.879772+0000","mapping_epoch":40,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267555,"num_objects":6658,"num_object_clones":0,"num_object_copies":13316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11459,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":59,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.619172+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:44.619172+0000","last_peered":"2024-09-11T01:20:44.619172+0000","last_clean":"2024-09-11T01:20:44.619172+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:44.619172+0000","last_undegraded":"2024-09-11T01:20:44.619172+0000","last_fullsized":"2024-09-11T01:20:44.619172+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'13758","reported_seq":14005,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.191771+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:47.191771+0000","last_peered":"2024-09-11T01:20:47.191771+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:47.191771+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28000256,"num_objects":6836,"num_object_clones":0,"num_object_copies":13672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6836,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10122,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":59,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790},{"shards":"0,2","objects":648}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"42'13403","reported_seq":13743,"reported_epoch":42,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.612089+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:45.612089+0000","last_peered":"2024-09-11T01:20:45.612089+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:45.612089+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3400","ondisk_log_start":"32'3400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591139,"num_objects":6737,"num_object_clones":0,"num_object_copies":13474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3264,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":6737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11172,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":332259,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":641},{"shards":"2","objects":2832}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":82,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.614736+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:44.614736+0000","last_peered":"2024-09-11T01:20:44.614736+0000","last_clean":"2024-09-11T01:20:44.614736+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:44.614736+0000","last_undegraded":"2024-09-11T01:20:44.614736+0000","last_fullsized":"2024-09-11T01:20:44.614736+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"42'13786","reported_seq":13928,"reported_epoch":42,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.088736+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:45.088736+0000","last_peered":"2024-09-11T01:20:45.088736+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:45.088736+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4154,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11423,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":675},{"shards":"2","objects":2624}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"41'13632","reported_seq":22094,"reported_epoch":42,"state":"peering","last_fresh":"2024-09-11T01:20:44.589340+0000","last_change":"2024-09-11T01:20:44.933985+0000","last_active":"2024-09-11T01:20:44.933985+0000","last_peered":"2024-09-11T01:20:44.589340+0000","last_clean":"2024-09-11T01:15:17.959258+0000","last_became_active":"2024-09-11T01:15:20.470479+0000","last_became_peered":"2024-09-11T01:15:20.470479+0000","last_unstale":"2024-09-11T01:20:44.589340+0000","last_undegraded":"2024-09-11T01:20:44.589340+0000","last_fullsized":"2024-09-11T01:15:19.985051+0000","mapping_epoch":42,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27525120,"num_objects":6720,"num_object_clones":0,"num_object_copies":13440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10322,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.630994+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:44.630994+0000","last_peered":"2024-09-11T01:20:44.630994+0000","last_clean":"2024-09-11T01:20:44.630994+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:44.630994+0000","last_undegraded":"2024-09-11T01:20:44.630994+0000","last_fullsized":"2024-09-11T01:20:44.630994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'13820","reported_seq":14090,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.304240+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:47.304240+0000","last_peered":"2024-09-11T01:20:47.304240+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:47.304240+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27959296,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10213,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2702},{"shards":"0,2","objects":699}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":59,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.618683+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:44.618683+0000","last_peered":"2024-09-11T01:20:44.618683+0000","last_clean":"2024-09-11T01:20:44.618683+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:44.618683+0000","last_undegraded":"2024-09-11T01:20:44.618683+0000","last_fullsized":"2024-09-11T01:20:44.618683+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'13813","reported_seq":14087,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.395969+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:47.395969+0000","last_peered":"2024-09-11T01:20:47.395969+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:47.395969+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10348,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709},{"shards":"0,2","objects":637}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.615159+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:44.615159+0000","last_peered":"2024-09-11T01:20:44.615159+0000","last_clean":"2024-09-11T01:20:44.615159+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:44.615159+0000","last_undegraded":"2024-09-11T01:20:44.615159+0000","last_fullsized":"2024-09-11T01:20:44.615159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.651606+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:44.651606+0000","last_peered":"2024-09-11T01:20:44.651606+0000","last_clean":"2024-09-11T01:20:44.651606+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:44.651606+0000","last_undegraded":"2024-09-11T01:20:44.651606+0000","last_fullsized":"2024-09-11T01:20:44.651606+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13553","reported_seq":13780,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.399797+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:47.399797+0000","last_peered":"2024-09-11T01:20:47.399797+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:47.399797+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26973126,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10210,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2479},{"shards":"0,2","objects":676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":441070326,"num_objects":107693,"num_object_clones":0,"num_object_copies":215386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40091,"num_objects_misplaced":641,"num_objects_unfound":0,"num_objects_dirty":107693,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":177548,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8902,"num_bytes_recovered":34280390,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1057890304,"data_stored":1057737640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":637932,"kb_used_data":349396,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93733908,"statfs":{"total":96636764160,"available":95983521792,"internally_reserved":0,"allocated":357781504,"data_stored":357504912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":295431854},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,2,3,2,0,0,0,0,0,1],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790018,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635088,"kb_used_data":296576,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93736752,"statfs":{"total":96636764160,"available":95986434048,"internally_reserved":0,"allocated":303693824,"data_stored":303382240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":763560,"kb_used_data":389004,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93608280,"statfs":{"total":96636764160,"available":95854878720,"internally_reserved":0,"allocated":398340096,"data_stored":398058263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2,0,0,0,0,8],"upper_bound":2048},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":397365248,"data_stored":397329118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303288320,"data_stored":303211988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":357236736,"data_stored":357196534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:52.468 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:53.031 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:53.033 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:53.061 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-11T01:20:52.557092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":432378633,"num_objects":105573,"num_object_clones":0,"num_object_copies":211146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40076,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":105573,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":179672,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9060,"num_bytes_recovered":34862041,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042632,"kb_used_data":1024404,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281072888,"statfs":{"total":289910292480,"available":287818637312,"internally_reserved":0,"allocated":1048989696,"data_stored":1048104000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1042598919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,8,4,3,0,0,0,2,0,9],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15884288,"num_objects":-3878,"num_object_clones":0,"num_object_copies":-7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-768,"num_objects_misplaced":426,"num_objects_unfound":0,"num_objects_dirty":-3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1212899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002653"},"pg_stats":[{"pgid":"2.f","version":"43'13727","reported_seq":13930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.414767+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:52.414767+0000","last_peered":"2024-09-11T01:20:52.414767+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:52.414767+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26669056,"num_objects":6511,"num_object_clones":0,"num_object_copies":13022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13727,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2453},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'13800","reported_seq":13880,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.554353+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:50.554353+0000","last_peered":"2024-09-11T01:20:50.554353+0000","last_clean":"2024-09-11T01:20:50.554353+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:50.554353+0000","last_undegraded":"2024-09-11T01:20:50.554353+0000","last_fullsized":"2024-09-11T01:20:50.554353+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27534278,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11898,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13514","reported_seq":13670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.554523+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:50.554523+0000","last_peered":"2024-09-11T01:20:50.554523+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:50.554523+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26751000,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11575,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":515},{"shards":"2","objects":2563}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13707","reported_seq":13752,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.559924+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:50.559924+0000","last_peered":"2024-09-11T01:20:50.559924+0000","last_clean":"2024-09-11T01:20:50.559924+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:50.559924+0000","last_undegraded":"2024-09-11T01:20:50.559924+0000","last_fullsized":"2024-09-11T01:20:50.559924+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13707,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13733","reported_seq":13855,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.407195+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:52.407195+0000","last_peered":"2024-09-11T01:20:52.407195+0000","last_clean":"2024-09-11T01:20:52.407195+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:52.407195+0000","last_undegraded":"2024-09-11T01:20:52.407195+0000","last_fullsized":"2024-09-11T01:20:52.407195+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26804707,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10531,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13714","reported_seq":13968,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.447758+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:52.447758+0000","last_peered":"2024-09-11T01:20:52.447758+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:52.447758+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26595811,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10202,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2439},{"shards":"0,2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14138","reported_seq":14403,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.428994+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:52.428994+0000","last_peered":"2024-09-11T01:20:52.428994+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:52.428994+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27337670,"num_objects":6676,"num_object_clones":0,"num_object_copies":13352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6676,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10891,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13828","reported_seq":13948,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.410582+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:52.410582+0000","last_peered":"2024-09-11T01:20:52.410582+0000","last_clean":"2024-09-11T01:20:52.410582+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:52.410582+0000","last_undegraded":"2024-09-11T01:20:52.410582+0000","last_fullsized":"2024-09-11T01:20:52.410582+0000","mapping_epoch":31,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13725","reported_seq":22068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.926112+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:49.926112+0000","last_peered":"2024-09-11T01:20:49.926112+0000","last_clean":"2024-09-11T01:20:49.926112+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:49.926112+0000","last_undegraded":"2024-09-11T01:20:49.926112+0000","last_fullsized":"2024-09-11T01:20:49.926112+0000","mapping_epoch":40,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722787,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11592,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'13899","reported_seq":14153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.447575+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:52.447575+0000","last_peered":"2024-09-11T01:20:52.447575+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:52.447575+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10263,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2653},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13508","reported_seq":14136,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.623370+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:50.623370+0000","last_peered":"2024-09-11T01:20:50.623370+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:50.623370+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27161059,"num_objects":6632,"num_object_clones":0,"num_object_copies":13264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":6632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11277,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":913891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":499},{"shards":"2","objects":2870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'13918","reported_seq":14064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.614482+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:50.614482+0000","last_peered":"2024-09-11T01:20:50.614482+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:50.614482+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11555,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":543},{"shards":"2","objects":2493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'13783","reported_seq":22254,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.867015+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:49.867015+0000","last_peered":"2024-09-11T01:20:49.867015+0000","last_clean":"2024-09-11T01:20:49.867015+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:49.867015+0000","last_undegraded":"2024-09-11T01:20:49.867015+0000","last_fullsized":"2024-09-11T01:20:49.867015+0000","mapping_epoch":42,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26906624,"num_objects":6569,"num_object_clones":0,"num_object_copies":13138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10473,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'13967","reported_seq":14238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.415265+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:52.415265+0000","last_peered":"2024-09-11T01:20:52.415265+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:52.415265+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2555},{"shards":"0,2","objects":552}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'13943","reported_seq":14221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.379392+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:52.379392+0000","last_peered":"2024-09-11T01:20:52.379392+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:52.379392+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2581},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13703","reported_seq":13930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.132091+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:52.132091+0000","last_peered":"2024-09-11T01:20:52.132091+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:52.132091+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26358726,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10360,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2329},{"shards":"0,2","objects":526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":432378614,"num_objects":105571,"num_object_clones":0,"num_object_copies":211142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40076,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":105571,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":179670,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9059,"num_bytes_recovered":34862022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046384640,"data_stored":1046231976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160607,"ondisk_log_size":160607,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638636,"kb_used_data":345948,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93733204,"statfs":{"total":96636764160,"available":95982800896,"internally_reserved":0,"allocated":354250752,"data_stored":353968924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790019,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642636,"kb_used_data":295804,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93729204,"statfs":{"total":96636764160,"available":95978704896,"internally_reserved":0,"allocated":302903296,"data_stored":302581261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761360,"kb_used_data":382652,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93610480,"statfs":{"total":96636764160,"available":95857131520,"internally_reserved":0,"allocated":391835648,"data_stored":391553815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,2,0,0,0,0,0,9],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":390758400,"data_stored":390722270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302452736,"data_stored":302376404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353173504,"data_stored":353133302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:53.063 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:53.638 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:53.641 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:53.663 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-11T01:20:52.557092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":432378633,"num_objects":105573,"num_object_clones":0,"num_object_copies":211146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40076,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":105573,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":179672,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9060,"num_bytes_recovered":34862041,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042632,"kb_used_data":1024404,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281072888,"statfs":{"total":289910292480,"available":287818637312,"internally_reserved":0,"allocated":1048989696,"data_stored":1048104000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1042598919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,8,4,3,0,0,0,2,0,9],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15884288,"num_objects":-3878,"num_object_clones":0,"num_object_copies":-7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-768,"num_objects_misplaced":426,"num_objects_unfound":0,"num_objects_dirty":-3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1212899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002653"},"pg_stats":[{"pgid":"2.f","version":"43'13727","reported_seq":13930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.414767+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:52.414767+0000","last_peered":"2024-09-11T01:20:52.414767+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:52.414767+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26669056,"num_objects":6511,"num_object_clones":0,"num_object_copies":13022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13727,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2453},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'13800","reported_seq":13880,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.554353+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:50.554353+0000","last_peered":"2024-09-11T01:20:50.554353+0000","last_clean":"2024-09-11T01:20:50.554353+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:50.554353+0000","last_undegraded":"2024-09-11T01:20:50.554353+0000","last_fullsized":"2024-09-11T01:20:50.554353+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27534278,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11898,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13514","reported_seq":13670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.554523+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:50.554523+0000","last_peered":"2024-09-11T01:20:50.554523+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:50.554523+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26751000,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11575,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":515},{"shards":"2","objects":2563}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13707","reported_seq":13752,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.559924+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:50.559924+0000","last_peered":"2024-09-11T01:20:50.559924+0000","last_clean":"2024-09-11T01:20:50.559924+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:50.559924+0000","last_undegraded":"2024-09-11T01:20:50.559924+0000","last_fullsized":"2024-09-11T01:20:50.559924+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13707,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13733","reported_seq":13855,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.407195+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:52.407195+0000","last_peered":"2024-09-11T01:20:52.407195+0000","last_clean":"2024-09-11T01:20:52.407195+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:52.407195+0000","last_undegraded":"2024-09-11T01:20:52.407195+0000","last_fullsized":"2024-09-11T01:20:52.407195+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26804707,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10531,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13714","reported_seq":13968,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.447758+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:52.447758+0000","last_peered":"2024-09-11T01:20:52.447758+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:52.447758+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26595811,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10202,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2439},{"shards":"0,2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14138","reported_seq":14403,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.428994+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:52.428994+0000","last_peered":"2024-09-11T01:20:52.428994+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:52.428994+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27337670,"num_objects":6676,"num_object_clones":0,"num_object_copies":13352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6676,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10891,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13828","reported_seq":13948,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.410582+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:52.410582+0000","last_peered":"2024-09-11T01:20:52.410582+0000","last_clean":"2024-09-11T01:20:52.410582+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:52.410582+0000","last_undegraded":"2024-09-11T01:20:52.410582+0000","last_fullsized":"2024-09-11T01:20:52.410582+0000","mapping_epoch":31,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13725","reported_seq":22068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.926112+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:49.926112+0000","last_peered":"2024-09-11T01:20:49.926112+0000","last_clean":"2024-09-11T01:20:49.926112+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:49.926112+0000","last_undegraded":"2024-09-11T01:20:49.926112+0000","last_fullsized":"2024-09-11T01:20:49.926112+0000","mapping_epoch":40,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722787,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11592,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'13899","reported_seq":14153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.447575+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:52.447575+0000","last_peered":"2024-09-11T01:20:52.447575+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:52.447575+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10263,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2653},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13508","reported_seq":14136,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.623370+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:50.623370+0000","last_peered":"2024-09-11T01:20:50.623370+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:50.623370+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27161059,"num_objects":6632,"num_object_clones":0,"num_object_copies":13264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":6632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11277,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":913891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":499},{"shards":"2","objects":2870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'13918","reported_seq":14064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.614482+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:50.614482+0000","last_peered":"2024-09-11T01:20:50.614482+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:50.614482+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11555,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":543},{"shards":"2","objects":2493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'13783","reported_seq":22254,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.867015+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:49.867015+0000","last_peered":"2024-09-11T01:20:49.867015+0000","last_clean":"2024-09-11T01:20:49.867015+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:49.867015+0000","last_undegraded":"2024-09-11T01:20:49.867015+0000","last_fullsized":"2024-09-11T01:20:49.867015+0000","mapping_epoch":42,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26906624,"num_objects":6569,"num_object_clones":0,"num_object_copies":13138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10473,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'13967","reported_seq":14238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.415265+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:52.415265+0000","last_peered":"2024-09-11T01:20:52.415265+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:52.415265+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2555},{"shards":"0,2","objects":552}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'13943","reported_seq":14221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.379392+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:52.379392+0000","last_peered":"2024-09-11T01:20:52.379392+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:52.379392+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2581},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13703","reported_seq":13930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.132091+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:52.132091+0000","last_peered":"2024-09-11T01:20:52.132091+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:52.132091+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26358726,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10360,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2329},{"shards":"0,2","objects":526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":432378614,"num_objects":105571,"num_object_clones":0,"num_object_copies":211142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40076,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":105571,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":179670,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9059,"num_bytes_recovered":34862022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046384640,"data_stored":1046231976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160607,"ondisk_log_size":160607,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638636,"kb_used_data":345948,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93733204,"statfs":{"total":96636764160,"available":95982800896,"internally_reserved":0,"allocated":354250752,"data_stored":353968924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790019,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642636,"kb_used_data":295804,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93729204,"statfs":{"total":96636764160,"available":95978704896,"internally_reserved":0,"allocated":302903296,"data_stored":302581261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761360,"kb_used_data":382652,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93610480,"statfs":{"total":96636764160,"available":95857131520,"internally_reserved":0,"allocated":391835648,"data_stored":391553815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,2,0,0,0,0,0,9],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":390758400,"data_stored":390722270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302452736,"data_stored":302376404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353173504,"data_stored":353133302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:53.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:53.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'13727", 'reported_seq': 13930, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.414767+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:52.414767+0000', 'last_peered': '2024-09-11T01:20:52.414767+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:52.414767+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26669056, 'num_objects': 6511, 'num_object_clones': 0, 'num_object_copies': 13022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4058, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13727, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 65, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2453}, {'shards': '0,2', 'objects': 515}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:53.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'13514", 'reported_seq': 13670, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:50.554523+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:50.554523+0000', 'last_peered': '2024-09-11T01:20:50.554523+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:50.554523+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'3500", 'ondisk_log_start': "32'3500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26751000, 'num_objects': 6532, 'num_object_clones': 0, 'num_object_copies': 13064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3970, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6532, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11575, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 37, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 515}, {'shards': '2', 'objects': 2563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:53.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:53.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'13714", 'reported_seq': 13968, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.447758+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:52.447758+0000', 'last_peered': '2024-09-11T01:20:52.447758+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:52.447758+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26595811, 'num_objects': 6494, 'num_object_clones': 0, 'num_object_copies': 12988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4056, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6494, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10202, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 60, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2439}, {'shards': '0,2', 'objects': 506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'14138", 'reported_seq': 14403, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.428994+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:52.428994+0000', 'last_peered': '2024-09-11T01:20:52.428994+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:52.428994+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "35'4100", 'ondisk_log_start': "35'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27337670, 'num_objects': 6676, 'num_object_clones': 0, 'num_object_copies': 13352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4217, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6676, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 10891, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2459}, {'shards': '0,2', 'objects': 554}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'13899", 'reported_seq': 14153, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.447575+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:52.447575+0000', 'last_peered': '2024-09-11T01:20:52.447575+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:52.447575+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "32'3800", 'ondisk_log_start': "32'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27422720, 'num_objects': 6695, 'num_object_clones': 0, 'num_object_copies': 13390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4044, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10263, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2653}, {'shards': '0,2', 'objects': 507}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'13508", 'reported_seq': 14136, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:50.623370+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:20:50.623370+0000', 'last_peered': '2024-09-11T01:20:50.623370+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:50.623370+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'3500", 'ondisk_log_start': "32'3500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27161059, 'num_objects': 6632, 'num_object_clones': 0, 'num_object_copies': 13264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 499, 'num_objects_unfound': 0, 'num_objects_dirty': 6632, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11277, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 266, 'num_bytes_recovered': 913891, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 499}, {'shards': '2', 'objects': 2870}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'13918", 'reported_seq': 14064, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:50.614482+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:50.614482+0000', 'last_peered': '2024-09-11T01:20:50.614482+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:50.614482+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "35'3900", 'ondisk_log_start': "35'3900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27213824, 'num_objects': 6644, 'num_object_clones': 0, 'num_object_copies': 13288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11555, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 543}, {'shards': '2', 'objects': 2493}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'13967", 'reported_seq': 14238, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.415265+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:52.415265+0000', 'last_peered': '2024-09-11T01:20:52.415265+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:52.415265+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "35'3900", 'ondisk_log_start': "35'3900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27357184, 'num_objects': 6679, 'num_object_clones': 0, 'num_object_copies': 13358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4127, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6679, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10360, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2555}, {'shards': '0,2', 'objects': 552}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:53.666 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'13943", 'reported_seq': 14221, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.379392+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:52.379392+0000', 'last_peered': '2024-09-11T01:20:52.379392+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:52.379392+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "35'3900", 'ondisk_log_start': "35'3900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27267072, 'num_objects': 6657, 'num_object_clones': 0, 'num_object_copies': 13314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10478, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2581}, {'shards': '0,2', 'objects': 507}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.666 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:53.666 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'13703", 'reported_seq': 13930, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.132091+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:52.132091+0000', 'last_peered': '2024-09-11T01:20:52.132091+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:52.132091+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26358726, 'num_objects': 6437, 'num_object_clones': 0, 'num_object_copies': 12874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4110, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6437, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10360, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 46, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2329}, {'shards': '0,2', 'objects': 526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:53.666 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:54.375 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:54.375 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40076/211146 objects degraded (18.980%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":454,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105573,"data_bytes":432378633,"bytes_used":2091655168,"bytes_avail":287818637312,"bytes_total":289910292480,"degraded_objects":40076,"degraded_total":211146,"degraded_ratio":0.18980231688026294,"misplaced_objects":499,"misplaced_total":211146,"misplaced_ratio":0.0023632936451554848,"recovering_objects_per_sec":42,"recovering_bytes_per_sec":151562,"recovering_keys_per_sec":0,"num_objects_recovered":342,"num_bytes_recovered":1212899,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":484},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:54.397 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:54.397 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:54.973 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:54.974 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:54.996 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-11T01:20:54.557804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":432378633,"num_objects":105573,"num_object_clones":0,"num_object_copies":211146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40076,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":105573,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":179672,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9060,"num_bytes_recovered":34862041,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042632,"kb_used_data":1024404,"kb_used_omap":12,"kb_used_meta":1018163,"kb_avail":281072888,"statfs":{"total":289910292480,"available":287818637312,"internally_reserved":0,"allocated":1048989696,"data_stored":1048104000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1042598919},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,8,4,3,0,0,0,2,0,9],"upper_bound":4096},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15884288,"num_objects":-3878,"num_object_clones":0,"num_object_copies":-7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-768,"num_objects_misplaced":426,"num_objects_unfound":0,"num_objects_dirty":-3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3878,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1212899,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.948288"},"pg_stats":[{"pgid":"2.f","version":"43'13727","reported_seq":13930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.414767+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:52.414767+0000","last_peered":"2024-09-11T01:20:52.414767+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:52.414767+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26669056,"num_objects":6511,"num_object_clones":0,"num_object_copies":13022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13727,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2453},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'13800","reported_seq":13880,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.554353+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:50.554353+0000","last_peered":"2024-09-11T01:20:50.554353+0000","last_clean":"2024-09-11T01:20:50.554353+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:50.554353+0000","last_undegraded":"2024-09-11T01:20:50.554353+0000","last_fullsized":"2024-09-11T01:20:50.554353+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27534278,"num_objects":6724,"num_object_clones":0,"num_object_copies":13448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6724,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11898,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13514","reported_seq":13670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.554523+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:50.554523+0000","last_peered":"2024-09-11T01:20:50.554523+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:50.554523+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26751000,"num_objects":6532,"num_object_clones":0,"num_object_copies":13064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3970,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11575,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":515},{"shards":"2","objects":2563}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13707","reported_seq":13752,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.559924+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:50.559924+0000","last_peered":"2024-09-11T01:20:50.559924+0000","last_clean":"2024-09-11T01:20:50.559924+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:50.559924+0000","last_undegraded":"2024-09-11T01:20:50.559924+0000","last_fullsized":"2024-09-11T01:20:50.559924+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13707,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13733","reported_seq":13855,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.407195+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:52.407195+0000","last_peered":"2024-09-11T01:20:52.407195+0000","last_clean":"2024-09-11T01:20:52.407195+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:52.407195+0000","last_undegraded":"2024-09-11T01:20:52.407195+0000","last_fullsized":"2024-09-11T01:20:52.407195+0000","mapping_epoch":31,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26804707,"num_objects":6545,"num_object_clones":0,"num_object_copies":13090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10531,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13714","reported_seq":13968,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.447758+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:52.447758+0000","last_peered":"2024-09-11T01:20:52.447758+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:52.447758+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26595811,"num_objects":6494,"num_object_clones":0,"num_object_copies":12988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10202,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2439},{"shards":"0,2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14138","reported_seq":14403,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.428994+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:52.428994+0000","last_peered":"2024-09-11T01:20:52.428994+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:52.428994+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27337670,"num_objects":6676,"num_object_clones":0,"num_object_copies":13352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6676,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10891,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2459},{"shards":"0,2","objects":554}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13828","reported_seq":13948,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.410582+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:52.410582+0000","last_peered":"2024-09-11T01:20:52.410582+0000","last_clean":"2024-09-11T01:20:52.410582+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:52.410582+0000","last_undegraded":"2024-09-11T01:20:52.410582+0000","last_fullsized":"2024-09-11T01:20:52.410582+0000","mapping_epoch":31,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27017216,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13725","reported_seq":22068,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.926112+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:49.926112+0000","last_peered":"2024-09-11T01:20:49.926112+0000","last_clean":"2024-09-11T01:20:49.926112+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:49.926112+0000","last_undegraded":"2024-09-11T01:20:49.926112+0000","last_fullsized":"2024-09-11T01:20:49.926112+0000","mapping_epoch":40,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722787,"num_objects":6525,"num_object_clones":0,"num_object_copies":13050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11592,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'13899","reported_seq":14153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.447575+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:52.447575+0000","last_peered":"2024-09-11T01:20:52.447575+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:52.447575+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27422720,"num_objects":6695,"num_object_clones":0,"num_object_copies":13390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10263,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2653},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13508","reported_seq":14136,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.623370+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:50.623370+0000","last_peered":"2024-09-11T01:20:50.623370+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:50.623370+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3500","ondisk_log_start":"32'3500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27161059,"num_objects":6632,"num_object_clones":0,"num_object_copies":13264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":6632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11277,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":913891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":499},{"shards":"2","objects":2870}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'13918","reported_seq":14064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.614482+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:50.614482+0000","last_peered":"2024-09-11T01:20:50.614482+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:50.614482+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11555,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":543},{"shards":"2","objects":2493}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'13783","reported_seq":22254,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.867015+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:49.867015+0000","last_peered":"2024-09-11T01:20:49.867015+0000","last_clean":"2024-09-11T01:20:49.867015+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:49.867015+0000","last_undegraded":"2024-09-11T01:20:49.867015+0000","last_fullsized":"2024-09-11T01:20:49.867015+0000","mapping_epoch":42,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26906624,"num_objects":6569,"num_object_clones":0,"num_object_copies":13138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10473,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'13967","reported_seq":14238,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.415265+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:52.415265+0000","last_peered":"2024-09-11T01:20:52.415265+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:52.415265+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27357184,"num_objects":6679,"num_object_clones":0,"num_object_copies":13358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4127,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2555},{"shards":"0,2","objects":552}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'13943","reported_seq":14221,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.379392+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:52.379392+0000","last_peered":"2024-09-11T01:20:52.379392+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:52.379392+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10478,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2581},{"shards":"0,2","objects":507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13703","reported_seq":13930,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.132091+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:52.132091+0000","last_peered":"2024-09-11T01:20:52.132091+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:52.132091+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26358726,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10360,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2329},{"shards":"0,2","objects":526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":432378614,"num_objects":105571,"num_object_clones":0,"num_object_copies":211142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40076,"num_objects_misplaced":499,"num_objects_unfound":0,"num_objects_dirty":105571,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":179670,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9059,"num_bytes_recovered":34862022,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1046384640,"data_stored":1046231976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160607,"ondisk_log_size":160607,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638636,"kb_used_data":345948,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93733204,"statfs":{"total":96636764160,"available":95982800896,"internally_reserved":0,"allocated":354250752,"data_stored":353968924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":299691694},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,1,1,0,0,0,2],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790019,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642636,"kb_used_data":295804,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93729204,"statfs":{"total":96636764160,"available":95978704896,"internally_reserved":0,"allocated":302903296,"data_stored":302581261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":761360,"kb_used_data":382652,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93610480,"statfs":{"total":96636764160,"available":95857131520,"internally_reserved":0,"allocated":391835648,"data_stored":391553815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,0,2,0,0,0,0,0,9],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":390758400,"data_stored":390722270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":302452736,"data_stored":302376404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":353173504,"data_stored":353133302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:57.998 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:58.578 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:58.580 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:58.603 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-11T01:20:58.559216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":423621385,"num_objects":103435,"num_object_clones":0,"num_object_copies":206870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40065,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":103435,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":181810,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9194,"num_bytes_recovered":35365849,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010000,"kb_used_data":1012576,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281105520,"statfs":{"total":289910292480,"available":287852052480,"internally_reserved":0,"allocated":1036877824,"data_stored":1035992128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1021299719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,1,8,5,5,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21639168,"num_objects":-5283,"num_object_clones":0,"num_object_copies":-10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":-5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004081"},"pg_stats":[{"pgid":"2.f","version":"43'13873","reported_seq":14080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.474907+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:57.474907+0000","last_peered":"2024-09-11T01:20:57.474907+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:57.474907+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26071040,"num_objects":6365,"num_object_clones":0,"num_object_copies":12730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2309},{"shards":"0,2","objects":369}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'13937","reported_seq":14017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.562811+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:55.562811+0000","last_peered":"2024-09-11T01:20:55.562811+0000","last_clean":"2024-09-11T01:20:55.562811+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:55.562811+0000","last_undegraded":"2024-09-11T01:20:55.562811+0000","last_fullsized":"2024-09-11T01:20:55.562811+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26973126,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12035,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13638","reported_seq":13797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.653563+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:55.653563+0000","last_peered":"2024-09-11T01:20:55.653563+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:55.653563+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243096,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11699,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":391},{"shards":"2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13841","reported_seq":13885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.652027+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:55.652027+0000","last_peered":"2024-09-11T01:20:55.652027+0000","last_clean":"2024-09-11T01:20:55.652027+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:55.652027+0000","last_undegraded":"2024-09-11T01:20:55.652027+0000","last_fullsized":"2024-09-11T01:20:55.652027+0000","mapping_epoch":31,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13841,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13858","reported_seq":13980,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.352334+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:57.352334+0000","last_peered":"2024-09-11T01:20:57.352334+0000","last_clean":"2024-09-11T01:20:57.352334+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:57.352334+0000","last_undegraded":"2024-09-11T01:20:57.352334+0000","last_fullsized":"2024-09-11T01:20:57.352334+0000","mapping_epoch":31,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292707,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10656,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13846","reported_seq":14103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.435357+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:57.435357+0000","last_peered":"2024-09-11T01:20:57.435357+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:57.435357+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26055139,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10334,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14263","reported_seq":14526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.466594+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:57.466594+0000","last_peered":"2024-09-11T01:20:57.466594+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:57.466594+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4200","ondisk_log_start":"35'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26825670,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11016,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2334},{"shards":"0,2","objects":429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13959","reported_seq":14079,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.426412+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:57.426412+0000","last_peered":"2024-09-11T01:20:57.426412+0000","last_clean":"2024-09-11T01:20:57.426412+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:57.426412+0000","last_undegraded":"2024-09-11T01:20:57.426412+0000","last_fullsized":"2024-09-11T01:20:57.426412+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10912,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13846","reported_seq":22189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.961416+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:54.961416+0000","last_peered":"2024-09-11T01:20:54.961416+0000","last_clean":"2024-09-11T01:20:54.961416+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:54.961416+0000","last_undegraded":"2024-09-11T01:20:54.961416+0000","last_fullsized":"2024-09-11T01:20:54.961416+0000","mapping_epoch":40,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26227171,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11713,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14027","reported_seq":14284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.369996+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:57.369996+0000","last_peered":"2024-09-11T01:20:57.369996+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:57.369996+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898432,"num_objects":6567,"num_object_clones":0,"num_object_copies":13134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10391,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2526},{"shards":"0,2","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13656","reported_seq":14530,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.617542+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:55.617542+0000","last_peered":"2024-09-11T01:20:55.617542+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:55.617542+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554851,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11425,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":389,"num_bytes_recovered":1417699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":376},{"shards":"2","objects":2845}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14045","reported_seq":14191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.635077+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:55.635077+0000","last_peered":"2024-09-11T01:20:55.635077+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:55.635077+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11682,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'13907","reported_seq":22378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.843043+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:54.843043+0000","last_peered":"2024-09-11T01:20:54.843043+0000","last_clean":"2024-09-11T01:20:54.843043+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:54.843043+0000","last_undegraded":"2024-09-11T01:20:54.843043+0000","last_fullsized":"2024-09-11T01:20:54.843043+0000","mapping_epoch":42,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26398720,"num_objects":6445,"num_object_clones":0,"num_object_copies":12890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10597,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14122","reported_seq":14397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.436705+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:57.436705+0000","last_peered":"2024-09-11T01:20:57.436705+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:57.436705+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14080","reported_seq":14362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.426263+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:57.426263+0000","last_peered":"2024-09-11T01:20:57.426263+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:57.426263+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26705920,"num_objects":6520,"num_object_clones":0,"num_object_copies":13040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10615,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13847","reported_seq":14076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.449920+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:57.449920+0000","last_peered":"2024-09-11T01:20:57.449920+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:57.449920+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25768902,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10504,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2186},{"shards":"0,2","objects":382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":423621366,"num_objects":103433,"num_object_clones":0,"num_object_copies":206866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40065,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":103433,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":181808,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9193,"num_bytes_recovered":35365830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1034211328,"data_stored":1034058664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621872,"kb_used_data":341664,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93749968,"statfs":{"total":96636764160,"available":95999967232,"internally_reserved":0,"allocated":349863936,"data_stored":349582108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629100,"kb_used_data":294748,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93742740,"statfs":{"total":96636764160,"available":95992565760,"internally_reserved":0,"allocated":301821952,"data_stored":301499917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759028,"kb_used_data":376164,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93612812,"statfs":{"total":96636764160,"available":95859519488,"internally_reserved":0,"allocated":385191936,"data_stored":384910103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,1,3,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384090112,"data_stored":384053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301350912,"data_stored":301274580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348770304,"data_stored":348730102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:58.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:59.181 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:59.182 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:20:59.212 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-11T01:20:58.559216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":423621385,"num_objects":103435,"num_object_clones":0,"num_object_copies":206870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40065,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":103435,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":181810,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9194,"num_bytes_recovered":35365849,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010000,"kb_used_data":1012576,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281105520,"statfs":{"total":289910292480,"available":287852052480,"internally_reserved":0,"allocated":1036877824,"data_stored":1035992128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1021299719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,1,8,5,5,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21639168,"num_objects":-5283,"num_object_clones":0,"num_object_copies":-10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":-5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004081"},"pg_stats":[{"pgid":"2.f","version":"43'13873","reported_seq":14080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.474907+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:57.474907+0000","last_peered":"2024-09-11T01:20:57.474907+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:57.474907+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26071040,"num_objects":6365,"num_object_clones":0,"num_object_copies":12730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2309},{"shards":"0,2","objects":369}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'13937","reported_seq":14017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.562811+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:55.562811+0000","last_peered":"2024-09-11T01:20:55.562811+0000","last_clean":"2024-09-11T01:20:55.562811+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:55.562811+0000","last_undegraded":"2024-09-11T01:20:55.562811+0000","last_fullsized":"2024-09-11T01:20:55.562811+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26973126,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12035,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13638","reported_seq":13797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.653563+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:55.653563+0000","last_peered":"2024-09-11T01:20:55.653563+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:55.653563+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243096,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11699,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":391},{"shards":"2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13841","reported_seq":13885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.652027+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:55.652027+0000","last_peered":"2024-09-11T01:20:55.652027+0000","last_clean":"2024-09-11T01:20:55.652027+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:55.652027+0000","last_undegraded":"2024-09-11T01:20:55.652027+0000","last_fullsized":"2024-09-11T01:20:55.652027+0000","mapping_epoch":31,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13841,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13858","reported_seq":13980,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.352334+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:57.352334+0000","last_peered":"2024-09-11T01:20:57.352334+0000","last_clean":"2024-09-11T01:20:57.352334+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:57.352334+0000","last_undegraded":"2024-09-11T01:20:57.352334+0000","last_fullsized":"2024-09-11T01:20:57.352334+0000","mapping_epoch":31,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292707,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10656,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13846","reported_seq":14103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.435357+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:57.435357+0000","last_peered":"2024-09-11T01:20:57.435357+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:57.435357+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26055139,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10334,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14263","reported_seq":14526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.466594+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:57.466594+0000","last_peered":"2024-09-11T01:20:57.466594+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:57.466594+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4200","ondisk_log_start":"35'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26825670,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11016,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2334},{"shards":"0,2","objects":429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13959","reported_seq":14079,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.426412+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:57.426412+0000","last_peered":"2024-09-11T01:20:57.426412+0000","last_clean":"2024-09-11T01:20:57.426412+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:57.426412+0000","last_undegraded":"2024-09-11T01:20:57.426412+0000","last_fullsized":"2024-09-11T01:20:57.426412+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10912,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13846","reported_seq":22189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.961416+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:54.961416+0000","last_peered":"2024-09-11T01:20:54.961416+0000","last_clean":"2024-09-11T01:20:54.961416+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:54.961416+0000","last_undegraded":"2024-09-11T01:20:54.961416+0000","last_fullsized":"2024-09-11T01:20:54.961416+0000","mapping_epoch":40,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26227171,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11713,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14027","reported_seq":14284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.369996+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:57.369996+0000","last_peered":"2024-09-11T01:20:57.369996+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:57.369996+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898432,"num_objects":6567,"num_object_clones":0,"num_object_copies":13134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10391,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2526},{"shards":"0,2","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13656","reported_seq":14530,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.617542+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:55.617542+0000","last_peered":"2024-09-11T01:20:55.617542+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:55.617542+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554851,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11425,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":389,"num_bytes_recovered":1417699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":376},{"shards":"2","objects":2845}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14045","reported_seq":14191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.635077+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:55.635077+0000","last_peered":"2024-09-11T01:20:55.635077+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:55.635077+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11682,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'13907","reported_seq":22378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.843043+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:54.843043+0000","last_peered":"2024-09-11T01:20:54.843043+0000","last_clean":"2024-09-11T01:20:54.843043+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:54.843043+0000","last_undegraded":"2024-09-11T01:20:54.843043+0000","last_fullsized":"2024-09-11T01:20:54.843043+0000","mapping_epoch":42,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26398720,"num_objects":6445,"num_object_clones":0,"num_object_copies":12890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10597,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14122","reported_seq":14397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.436705+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:57.436705+0000","last_peered":"2024-09-11T01:20:57.436705+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:57.436705+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14080","reported_seq":14362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.426263+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:57.426263+0000","last_peered":"2024-09-11T01:20:57.426263+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:57.426263+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26705920,"num_objects":6520,"num_object_clones":0,"num_object_copies":13040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10615,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13847","reported_seq":14076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.449920+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:57.449920+0000","last_peered":"2024-09-11T01:20:57.449920+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:57.449920+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25768902,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10504,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2186},{"shards":"0,2","objects":382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":423621366,"num_objects":103433,"num_object_clones":0,"num_object_copies":206866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40065,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":103433,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":181808,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9193,"num_bytes_recovered":35365830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1034211328,"data_stored":1034058664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621872,"kb_used_data":341664,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93749968,"statfs":{"total":96636764160,"available":95999967232,"internally_reserved":0,"allocated":349863936,"data_stored":349582108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629100,"kb_used_data":294748,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93742740,"statfs":{"total":96636764160,"available":95992565760,"internally_reserved":0,"allocated":301821952,"data_stored":301499917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759028,"kb_used_data":376164,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93612812,"statfs":{"total":96636764160,"available":95859519488,"internally_reserved":0,"allocated":385191936,"data_stored":384910103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,1,3,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384090112,"data_stored":384053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301350912,"data_stored":301274580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348770304,"data_stored":348730102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:59.213 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'13873", 'reported_seq': 14080, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.474907+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:20:57.474907+0000', 'last_peered': '2024-09-11T01:20:57.474907+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:20:57.474907+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "32'3800", 'ondisk_log_start': "32'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26071040, 'num_objects': 6365, 'num_object_clones': 0, 'num_object_copies': 12730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4056, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6365, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13873, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 67, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2309}, {'shards': '0,2', 'objects': 369}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'13638", 'reported_seq': 13797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:55.653563+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:20:55.653563+0000', 'last_peered': '2024-09-11T01:20:55.653563+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:20:55.653563+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26243096, 'num_objects': 6408, 'num_object_clones': 0, 'num_object_copies': 12816, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3968, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6408, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11699, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 391}, {'shards': '2', 'objects': 2441}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'13846", 'reported_seq': 14103, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.435357+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:20:57.435357+0000', 'last_peered': '2024-09-11T01:20:57.435357+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:20:57.435357+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "32'3800", 'ondisk_log_start': "32'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26055139, 'num_objects': 6362, 'num_object_clones': 0, 'num_object_copies': 12724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4055, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6362, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10334, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2308}, {'shards': '0,2', 'objects': 374}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'14263", 'reported_seq': 14526, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.466594+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:20:57.466594+0000', 'last_peered': '2024-09-11T01:20:57.466594+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:20:57.466594+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "35'4200", 'ondisk_log_start': "35'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26825670, 'num_objects': 6551, 'num_object_clones': 0, 'num_object_copies': 13102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4217, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6551, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11016, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2334}, {'shards': '0,2', 'objects': 429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:59.214 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14027", 'reported_seq': 14284, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.369996+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:20:57.369996+0000', 'last_peered': '2024-09-11T01:20:57.369996+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:20:57.369996+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "35'4000", 'ondisk_log_start': "35'4000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26898432, 'num_objects': 6567, 'num_object_clones': 0, 'num_object_copies': 13134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4043, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10391, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 64, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2526}, {'shards': '0,2', 'objects': 379}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'13656", 'reported_seq': 14530, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:55.617542+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:20:55.617542+0000', 'last_peered': '2024-09-11T01:20:55.617542+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:20:55.617542+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'3600", 'ondisk_log_start': "32'3600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26554851, 'num_objects': 6484, 'num_object_clones': 0, 'num_object_copies': 12968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 376, 'num_objects_unfound': 0, 'num_objects_dirty': 6484, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11425, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 389, 'num_bytes_recovered': 1417699, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 376}, {'shards': '2', 'objects': 2845}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'14045", 'reported_seq': 14191, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:55.635077+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:20:55.635077+0000', 'last_peered': '2024-09-11T01:20:55.635077+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:20:55.635077+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "35'4000", 'ondisk_log_start': "35'4000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26693632, 'num_objects': 6517, 'num_object_clones': 0, 'num_object_copies': 13034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11682, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 416}, {'shards': '2', 'objects': 2366}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'14122", 'reported_seq': 14397, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.436705+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:20:57.436705+0000', 'last_peered': '2024-09-11T01:20:57.436705+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:20:57.436705+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "35'4100", 'ondisk_log_start': "35'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26722304, 'num_objects': 6524, 'num_object_clones': 0, 'num_object_copies': 13048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4125, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10515, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 70, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2402}, {'shards': '0,2', 'objects': 397}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14080", 'reported_seq': 14362, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.426263+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:20:57.426263+0000', 'last_peered': '2024-09-11T01:20:57.426263+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:20:57.426263+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "35'4000", 'ondisk_log_start': "35'4000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26705920, 'num_objects': 6520, 'num_object_clones': 0, 'num_object_copies': 13040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4076, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10615, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 74, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2446}, {'shards': '0,2', 'objects': 370}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:59.215 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'13847", 'reported_seq': 14076, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:57.449920+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:20:57.449920+0000', 'last_peered': '2024-09-11T01:20:57.449920+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:20:57.449920+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "32'3800", 'ondisk_log_start': "32'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25768902, 'num_objects': 6293, 'num_object_clones': 0, 'num_object_copies': 12586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4109, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6293, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10504, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 47, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2186}, {'shards': '0,2', 'objects': 382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:59.216 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:59.909 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:20:59.909 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40065/206870 objects degraded (19.367%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":460,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103435,"data_bytes":423621385,"bytes_used":2058240000,"bytes_avail":287852052480,"bytes_total":289910292480,"degraded_objects":40065,"degraded_total":206870,"degraded_ratio":0.19367235461884275,"misplaced_objects":376,"misplaced_total":206870,"misplaced_ratio":0.0018175665877120898,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":90422,"recovering_keys_per_sec":0,"num_objects_recovered":315,"num_bytes_recovered":1085440,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:59.935 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:59.935 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:00.511 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:00.513 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:00.534 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":266,"stamp":"2024-09-11T01:20:58.559216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":423621385,"num_objects":103435,"num_object_clones":0,"num_object_copies":206870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40065,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":103435,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":181810,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9194,"num_bytes_recovered":35365849,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010000,"kb_used_data":1012576,"kb_used_omap":12,"kb_used_meta":997363,"kb_avail":281105520,"statfs":{"total":289910292480,"available":287852052480,"internally_reserved":0,"allocated":1036877824,"data_stored":1035992128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1021299719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,1,8,5,5,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21639168,"num_objects":-5283,"num_object_clones":0,"num_object_copies":-10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-50,"num_objects_misplaced":-265,"num_objects_unfound":0,"num_objects_dirty":-5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5283,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":1085440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004081"},"pg_stats":[{"pgid":"2.f","version":"43'13873","reported_seq":14080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.474907+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:20:57.474907+0000","last_peered":"2024-09-11T01:20:57.474907+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:20:57.474907+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26071040,"num_objects":6365,"num_object_clones":0,"num_object_copies":12730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4056,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13873,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":67,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2309},{"shards":"0,2","objects":369}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'13937","reported_seq":14017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.562811+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:20:55.562811+0000","last_peered":"2024-09-11T01:20:55.562811+0000","last_clean":"2024-09-11T01:20:55.562811+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:20:55.562811+0000","last_undegraded":"2024-09-11T01:20:55.562811+0000","last_fullsized":"2024-09-11T01:20:55.562811+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26973126,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12035,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13638","reported_seq":13797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.653563+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:20:55.653563+0000","last_peered":"2024-09-11T01:20:55.653563+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:20:55.653563+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26243096,"num_objects":6408,"num_object_clones":0,"num_object_copies":12816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11699,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":391},{"shards":"2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13841","reported_seq":13885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.652027+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:20:55.652027+0000","last_peered":"2024-09-11T01:20:55.652027+0000","last_clean":"2024-09-11T01:20:55.652027+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:20:55.652027+0000","last_undegraded":"2024-09-11T01:20:55.652027+0000","last_fullsized":"2024-09-11T01:20:55.652027+0000","mapping_epoch":31,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26710016,"num_objects":6521,"num_object_clones":0,"num_object_copies":13042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13841,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13858","reported_seq":13980,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.352334+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:20:57.352334+0000","last_peered":"2024-09-11T01:20:57.352334+0000","last_clean":"2024-09-11T01:20:57.352334+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:20:57.352334+0000","last_undegraded":"2024-09-11T01:20:57.352334+0000","last_fullsized":"2024-09-11T01:20:57.352334+0000","mapping_epoch":31,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26292707,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10656,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13846","reported_seq":14103,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.435357+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:20:57.435357+0000","last_peered":"2024-09-11T01:20:57.435357+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:20:57.435357+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26055139,"num_objects":6362,"num_object_clones":0,"num_object_copies":12724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6362,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10334,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2308},{"shards":"0,2","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14263","reported_seq":14526,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.466594+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:20:57.466594+0000","last_peered":"2024-09-11T01:20:57.466594+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:20:57.466594+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"35'4200","ondisk_log_start":"35'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26825670,"num_objects":6551,"num_object_clones":0,"num_object_copies":13102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4217,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6551,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11016,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2334},{"shards":"0,2","objects":429}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'13959","reported_seq":14079,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.426412+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:20:57.426412+0000","last_peered":"2024-09-11T01:20:57.426412+0000","last_clean":"2024-09-11T01:20:57.426412+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:20:57.426412+0000","last_undegraded":"2024-09-11T01:20:57.426412+0000","last_fullsized":"2024-09-11T01:20:57.426412+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26480640,"num_objects":6465,"num_object_clones":0,"num_object_copies":12930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10912,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13846","reported_seq":22189,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.961416+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:54.961416+0000","last_peered":"2024-09-11T01:20:54.961416+0000","last_clean":"2024-09-11T01:20:54.961416+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:54.961416+0000","last_undegraded":"2024-09-11T01:20:54.961416+0000","last_fullsized":"2024-09-11T01:20:54.961416+0000","mapping_epoch":40,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26227171,"num_objects":6404,"num_object_clones":0,"num_object_copies":12808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11713,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14027","reported_seq":14284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.369996+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:20:57.369996+0000","last_peered":"2024-09-11T01:20:57.369996+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:20:57.369996+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898432,"num_objects":6567,"num_object_clones":0,"num_object_copies":13134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4043,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10391,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2526},{"shards":"0,2","objects":379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13656","reported_seq":14530,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.617542+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:20:55.617542+0000","last_peered":"2024-09-11T01:20:55.617542+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:20:55.617542+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3600","ondisk_log_start":"32'3600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554851,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11425,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":389,"num_bytes_recovered":1417699,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":376},{"shards":"2","objects":2845}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14045","reported_seq":14191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.635077+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:20:55.635077+0000","last_peered":"2024-09-11T01:20:55.635077+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:20:55.635077+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26693632,"num_objects":6517,"num_object_clones":0,"num_object_copies":13034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11682,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":416},{"shards":"2","objects":2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'13907","reported_seq":22378,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.843043+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:54.843043+0000","last_peered":"2024-09-11T01:20:54.843043+0000","last_clean":"2024-09-11T01:20:54.843043+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:54.843043+0000","last_undegraded":"2024-09-11T01:20:54.843043+0000","last_fullsized":"2024-09-11T01:20:54.843043+0000","mapping_epoch":42,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26398720,"num_objects":6445,"num_object_clones":0,"num_object_copies":12890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10597,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14122","reported_seq":14397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.436705+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:20:57.436705+0000","last_peered":"2024-09-11T01:20:57.436705+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:20:57.436705+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26722304,"num_objects":6524,"num_object_clones":0,"num_object_copies":13048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4125,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10515,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14080","reported_seq":14362,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.426263+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:20:57.426263+0000","last_peered":"2024-09-11T01:20:57.426263+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:20:57.426263+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26705920,"num_objects":6520,"num_object_clones":0,"num_object_copies":13040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4076,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10615,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":370}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13847","reported_seq":14076,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.449920+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:20:57.449920+0000","last_peered":"2024-09-11T01:20:57.449920+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:20:57.449920+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"32'3800","ondisk_log_start":"32'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25768902,"num_objects":6293,"num_object_clones":0,"num_object_copies":12586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6293,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10504,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2186},{"shards":"0,2","objects":382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":423621366,"num_objects":103433,"num_object_clones":0,"num_object_copies":206866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40065,"num_objects_misplaced":376,"num_objects_unfound":0,"num_objects_dirty":103433,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":181808,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9193,"num_bytes_recovered":35365830,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1034211328,"data_stored":1034058664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621872,"kb_used_data":341664,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93749968,"statfs":{"total":96636764160,"available":95999967232,"internally_reserved":0,"allocated":349863936,"data_stored":349582108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":286912174},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790020,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629100,"kb_used_data":294748,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93742740,"statfs":{"total":96636764160,"available":95992565760,"internally_reserved":0,"allocated":301821952,"data_stored":301499917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":759028,"kb_used_data":376164,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93612812,"statfs":{"total":96636764160,"available":95859519488,"internally_reserved":0,"allocated":385191936,"data_stored":384910103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,4,1,3,0,0,0,0,0,4],"upper_bound":8192},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":384090112,"data_stored":384053982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301350912,"data_stored":301274580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":348770304,"data_stored":348730102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:03.536 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:04.088 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:04.088 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:04.121 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-11T01:21:02.561969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":414487305,"num_objects":101205,"num_object_clones":0,"num_object_copies":202410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40054,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":101205,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":184040,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9305,"num_bytes_recovered":35775449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160978,"ondisk_log_size":160978,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2020032,"kb_used_data":1000396,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281095488,"statfs":{"total":289910292480,"available":287841779712,"internally_reserved":0,"allocated":1024405504,"data_stored":1023519808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1044040711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,9,6,8,1,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25419776,"num_objects":-6206,"num_object_clones":0,"num_object_copies":-12412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-37,"num_objects_misplaced":-365,"num_objects_unfound":0,"num_objects_dirty":-6206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6206,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005340"},"pg_stats":[{"pgid":"2.f","version":"43'14010","reported_seq":14220,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.417598+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:02.417598+0000","last_peered":"2024-09-11T01:21:02.417598+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:02.417598+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25509888,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14010,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2173},{"shards":"0,2","objects":232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14087","reported_seq":14167,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.597596+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:00.597596+0000","last_peered":"2024-09-11T01:21:00.597596+0000","last_clean":"2024-09-11T01:21:00.597596+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:00.597596+0000","last_undegraded":"2024-09-11T01:21:00.597596+0000","last_fullsized":"2024-09-11T01:21:00.597596+0000","mapping_epoch":31,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26358726,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12185,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13768","reported_seq":13928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.675129+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:00.675129+0000","last_peered":"2024-09-11T01:21:00.675129+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:00.675129+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710616,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11829,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":261},{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13977","reported_seq":14022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.653564+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:00.653564+0000","last_peered":"2024-09-11T01:21:00.653564+0000","last_clean":"2024-09-11T01:21:00.653564+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:00.653564+0000","last_undegraded":"2024-09-11T01:21:00.653564+0000","last_fullsized":"2024-09-11T01:21:00.653564+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26152960,"num_objects":6385,"num_object_clones":0,"num_object_copies":12770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13977,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13979","reported_seq":14101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.482085+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:02.482085+0000","last_peered":"2024-09-11T01:21:02.482085+0000","last_clean":"2024-09-11T01:21:02.482085+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:02.482085+0000","last_undegraded":"2024-09-11T01:21:02.482085+0000","last_fullsized":"2024-09-11T01:21:02.482085+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25797091,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10777,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13981","reported_seq":14240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.470156+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:02.470156+0000","last_peered":"2024-09-11T01:21:02.470156+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:02.470156+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25502179,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10469,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2174},{"shards":"0,2","objects":239}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14431","reported_seq":14705,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.404412+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:02.404412+0000","last_peered":"2024-09-11T01:21:02.404412+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:02.404412+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26137542,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11184,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14105","reported_seq":14225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.433120+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:02.433120+0000","last_peered":"2024-09-11T01:21:02.433120+0000","last_clean":"2024-09-11T01:21:02.433120+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:02.433120+0000","last_undegraded":"2024-09-11T01:21:02.433120+0000","last_fullsized":"2024-09-11T01:21:02.433120+0000","mapping_epoch":31,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25882624,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11058,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13979","reported_seq":22322,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.949551+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:59.949551+0000","last_peered":"2024-09-11T01:20:59.949551+0000","last_clean":"2024-09-11T01:20:59.949551+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:59.949551+0000","last_undegraded":"2024-09-11T01:20:59.949551+0000","last_fullsized":"2024-09-11T01:20:59.949551+0000","mapping_epoch":40,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25682403,"num_objects":6271,"num_object_clones":0,"num_object_copies":12542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11846,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14167","reported_seq":14426,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.427145+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:02.427145+0000","last_peered":"2024-09-11T01:21:02.427145+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:02.427145+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26324992,"num_objects":6427,"num_object_clones":0,"num_object_copies":12854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10531,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13796","reported_seq":14870,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.666507+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:00.666507+0000","last_peered":"2024-09-11T01:21:00.666507+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:00.666507+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25981411,"num_objects":6344,"num_object_clones":0,"num_object_copies":12688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":6344,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11565,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1827299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":276},{"shards":"2","objects":2805}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14184","reported_seq":14330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.555795+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:00.555795+0000","last_peered":"2024-09-11T01:21:00.555795+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:00.555795+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11821,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":277},{"shards":"2","objects":2227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14047","reported_seq":22518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.988931+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:59.988931+0000","last_peered":"2024-09-11T01:20:59.988931+0000","last_clean":"2024-09-11T01:20:59.988931+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:59.988931+0000","last_undegraded":"2024-09-11T01:20:59.988931+0000","last_fullsized":"2024-09-11T01:20:59.988931+0000","mapping_epoch":42,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10737,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14257","reported_seq":14536,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.445457+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:02.445457+0000","last_peered":"2024-09-11T01:21:02.445457+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:02.445457+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"36'4200","ondisk_log_start":"36'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10650,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2269},{"shards":"0,2","objects":262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14222","reported_seq":14506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.471552+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:02.471552+0000","last_peered":"2024-09-11T01:21:02.471552+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:02.471552+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10757,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2305},{"shards":"0,2","objects":228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13985","reported_seq":14216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.483730+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:02.483730+0000","last_peered":"2024-09-11T01:21:02.483730+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:02.483730+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25203654,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10642,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2049},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":414487286,"num_objects":101203,"num_object_clones":0,"num_object_copies":202406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40054,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":101203,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":184038,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9304,"num_bytes_recovered":35775430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021722624,"data_stored":1021569960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160975,"ondisk_log_size":160975,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639728,"kb_used_data":337312,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93732112,"statfs":{"total":96636764160,"available":95981682688,"internally_reserved":0,"allocated":345407488,"data_stored":345125660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636200,"kb_used_data":293528,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93735640,"statfs":{"total":96636764160,"available":95985295360,"internally_reserved":0,"allocated":300572672,"data_stored":300250637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744104,"kb_used_data":369556,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93627736,"statfs":{"total":96636764160,"available":95874801664,"internally_reserved":0,"allocated":378425344,"data_stored":378143511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3,1,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377200640,"data_stored":377164510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300077056,"data_stored":300000724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344444928,"data_stored":344404726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:04.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:04.684 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:04.687 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:04.717 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-11T01:21:04.562946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":414487305,"num_objects":101205,"num_object_clones":0,"num_object_copies":202410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40054,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":101205,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":184040,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9305,"num_bytes_recovered":35775449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160978,"ondisk_log_size":160978,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2020032,"kb_used_data":1000396,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281095488,"statfs":{"total":289910292480,"available":287841779712,"internally_reserved":0,"allocated":1024405504,"data_stored":1023519808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1044040711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,9,6,8,1,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17891328,"num_objects":-4368,"num_object_clones":0,"num_object_copies":-8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":-4368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4368,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005853"},"pg_stats":[{"pgid":"2.f","version":"43'14010","reported_seq":14220,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.417598+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:02.417598+0000","last_peered":"2024-09-11T01:21:02.417598+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:02.417598+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25509888,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14010,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2173},{"shards":"0,2","objects":232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14087","reported_seq":14167,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.597596+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:00.597596+0000","last_peered":"2024-09-11T01:21:00.597596+0000","last_clean":"2024-09-11T01:21:00.597596+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:00.597596+0000","last_undegraded":"2024-09-11T01:21:00.597596+0000","last_fullsized":"2024-09-11T01:21:00.597596+0000","mapping_epoch":31,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26358726,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12185,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13768","reported_seq":13928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.675129+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:00.675129+0000","last_peered":"2024-09-11T01:21:00.675129+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:00.675129+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710616,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11829,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":261},{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13977","reported_seq":14022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.653564+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:00.653564+0000","last_peered":"2024-09-11T01:21:00.653564+0000","last_clean":"2024-09-11T01:21:00.653564+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:00.653564+0000","last_undegraded":"2024-09-11T01:21:00.653564+0000","last_fullsized":"2024-09-11T01:21:00.653564+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26152960,"num_objects":6385,"num_object_clones":0,"num_object_copies":12770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13977,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13979","reported_seq":14101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.482085+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:02.482085+0000","last_peered":"2024-09-11T01:21:02.482085+0000","last_clean":"2024-09-11T01:21:02.482085+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:02.482085+0000","last_undegraded":"2024-09-11T01:21:02.482085+0000","last_fullsized":"2024-09-11T01:21:02.482085+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25797091,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10777,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13981","reported_seq":14240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.470156+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:02.470156+0000","last_peered":"2024-09-11T01:21:02.470156+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:02.470156+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25502179,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10469,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2174},{"shards":"0,2","objects":239}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14431","reported_seq":14705,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.404412+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:02.404412+0000","last_peered":"2024-09-11T01:21:02.404412+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:02.404412+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26137542,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11184,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14105","reported_seq":14225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.433120+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:02.433120+0000","last_peered":"2024-09-11T01:21:02.433120+0000","last_clean":"2024-09-11T01:21:02.433120+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:02.433120+0000","last_undegraded":"2024-09-11T01:21:02.433120+0000","last_fullsized":"2024-09-11T01:21:02.433120+0000","mapping_epoch":31,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25882624,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11058,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13979","reported_seq":22322,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.949551+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:59.949551+0000","last_peered":"2024-09-11T01:20:59.949551+0000","last_clean":"2024-09-11T01:20:59.949551+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:59.949551+0000","last_undegraded":"2024-09-11T01:20:59.949551+0000","last_fullsized":"2024-09-11T01:20:59.949551+0000","mapping_epoch":40,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25682403,"num_objects":6271,"num_object_clones":0,"num_object_copies":12542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11846,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14167","reported_seq":14426,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.427145+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:02.427145+0000","last_peered":"2024-09-11T01:21:02.427145+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:02.427145+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26324992,"num_objects":6427,"num_object_clones":0,"num_object_copies":12854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10531,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13796","reported_seq":14870,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.666507+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:00.666507+0000","last_peered":"2024-09-11T01:21:00.666507+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:00.666507+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25981411,"num_objects":6344,"num_object_clones":0,"num_object_copies":12688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":6344,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11565,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1827299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":276},{"shards":"2","objects":2805}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14184","reported_seq":14330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.555795+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:00.555795+0000","last_peered":"2024-09-11T01:21:00.555795+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:00.555795+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11821,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":277},{"shards":"2","objects":2227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14047","reported_seq":22518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.988931+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:59.988931+0000","last_peered":"2024-09-11T01:20:59.988931+0000","last_clean":"2024-09-11T01:20:59.988931+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:59.988931+0000","last_undegraded":"2024-09-11T01:20:59.988931+0000","last_fullsized":"2024-09-11T01:20:59.988931+0000","mapping_epoch":42,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10737,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14257","reported_seq":14536,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.445457+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:02.445457+0000","last_peered":"2024-09-11T01:21:02.445457+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:02.445457+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"36'4200","ondisk_log_start":"36'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10650,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2269},{"shards":"0,2","objects":262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14222","reported_seq":14506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.471552+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:02.471552+0000","last_peered":"2024-09-11T01:21:02.471552+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:02.471552+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10757,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2305},{"shards":"0,2","objects":228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13985","reported_seq":14216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.483730+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:02.483730+0000","last_peered":"2024-09-11T01:21:02.483730+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:02.483730+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25203654,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10642,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2049},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":414487286,"num_objects":101203,"num_object_clones":0,"num_object_copies":202406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40054,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":101203,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":184038,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9304,"num_bytes_recovered":35775430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021722624,"data_stored":1021569960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160975,"ondisk_log_size":160975,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639728,"kb_used_data":337312,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93732112,"statfs":{"total":96636764160,"available":95981682688,"internally_reserved":0,"allocated":345407488,"data_stored":345125660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636200,"kb_used_data":293528,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93735640,"statfs":{"total":96636764160,"available":95985295360,"internally_reserved":0,"allocated":300572672,"data_stored":300250637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744104,"kb_used_data":369556,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93627736,"statfs":{"total":96636764160,"available":95874801664,"internally_reserved":0,"allocated":378425344,"data_stored":378143511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3,1,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377200640,"data_stored":377164510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300077056,"data_stored":300000724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344444928,"data_stored":344404726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14010", 'reported_seq': 14220, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.417598+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:02.417598+0000', 'last_peered': '2024-09-11T01:21:02.417598+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:02.417598+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "35'4000", 'ondisk_log_start': "35'4000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25509888, 'num_objects': 6228, 'num_object_clones': 0, 'num_object_copies': 12456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4055, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14010, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2173}, {'shards': '0,2', 'objects': 232}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'13768", 'reported_seq': 13928, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:00.675129+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:00.675129+0000', 'last_peered': '2024-09-11T01:21:00.675129+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:00.675129+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25710616, 'num_objects': 6278, 'num_object_clones': 0, 'num_object_copies': 12556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3968, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6278, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11829, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 261}, {'shards': '2', 'objects': 2311}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'13981", 'reported_seq': 14240, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.470156+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:02.470156+0000', 'last_peered': '2024-09-11T01:21:02.470156+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:02.470156+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "35'3900", 'ondisk_log_start': "35'3900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25502179, 'num_objects': 6227, 'num_object_clones': 0, 'num_object_copies': 12454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4054, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6227, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10469, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 62, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2174}, {'shards': '0,2', 'objects': 239}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:04.719 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'14431", 'reported_seq': 14705, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.404412+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:02.404412+0000', 'last_peered': '2024-09-11T01:21:02.404412+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:02.404412+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26137542, 'num_objects': 6383, 'num_object_clones': 0, 'num_object_copies': 12766, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4213, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6383, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11184, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 70, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2170}, {'shards': '0,2', 'objects': 261}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14167", 'reported_seq': 14426, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.427145+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:02.427145+0000', 'last_peered': '2024-09-11T01:21:02.427145+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:02.427145+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "36'4100", 'ondisk_log_start': "36'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26324992, 'num_objects': 6427, 'num_object_clones': 0, 'num_object_copies': 12854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4042, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10531, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 65, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2387}, {'shards': '0,2', 'objects': 239}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'13796", 'reported_seq': 14870, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:00.666507+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:00.666507+0000', 'last_peered': '2024-09-11T01:21:00.666507+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:00.666507+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "32'3700", 'ondisk_log_start': "32'3700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25981411, 'num_objects': 6344, 'num_object_clones': 0, 'num_object_copies': 12688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 276, 'num_objects_unfound': 0, 'num_objects_dirty': 6344, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11565, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 489, 'num_bytes_recovered': 1827299, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 276}, {'shards': '2', 'objects': 2805}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'14184", 'reported_seq': 14330, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:00.555795+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:00.555795+0000', 'last_peered': '2024-09-11T01:21:00.555795+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:00.555795+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "35'4100", 'ondisk_log_start': "35'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26124288, 'num_objects': 6378, 'num_object_clones': 0, 'num_object_copies': 12756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4153, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11821, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 26, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 277}, {'shards': '2', 'objects': 2227}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'14257", 'reported_seq': 14536, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.445457+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:02.445457+0000', 'last_peered': '2024-09-11T01:21:02.445457+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:02.445457+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "36'4200", 'ondisk_log_start': "36'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26169344, 'num_objects': 6389, 'num_object_clones': 0, 'num_object_copies': 12778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6389, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10650, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2269}, {'shards': '0,2', 'objects': 262}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:04.720 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14222", 'reported_seq': 14506, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.471552+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:02.471552+0000', 'last_peered': '2024-09-11T01:21:02.471552+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:02.471552+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26124288, 'num_objects': 6378, 'num_object_clones': 0, 'num_object_copies': 12756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4075, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6378, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10757, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2305}, {'shards': '0,2', 'objects': 228}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.721 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:04.721 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'13985", 'reported_seq': 14216, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.483730+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:02.483730+0000', 'last_peered': '2024-09-11T01:21:02.483730+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:02.483730+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "35'3900", 'ondisk_log_start': "35'3900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25203654, 'num_objects': 6155, 'num_object_clones': 0, 'num_object_copies': 12310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4108, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6155, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10642, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2049}, {'shards': '0,2', 'objects': 244}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:04.721 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:05.496 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:05.497 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40054/202410 objects degraded (19.789%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":465,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101205,"data_bytes":414487305,"bytes_used":2068512768,"bytes_avail":287841779712,"bytes_total":289910292480,"degraded_objects":40054,"degraded_total":202410,"degraded_ratio":0.19788547996640482,"misplaced_objects":276,"misplaced_total":202410,"misplaced_ratio":0.0013635689936267971,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":76080,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":913408,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":363},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:05.525 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:05.526 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:06.096 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:06.096 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:06.134 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-11T01:21:04.562946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":414487305,"num_objects":101205,"num_object_clones":0,"num_object_copies":202410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40054,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":101205,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":184040,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9305,"num_bytes_recovered":35775449,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160978,"ondisk_log_size":160978,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2020032,"kb_used_data":1000396,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":281095488,"statfs":{"total":289910292480,"available":287841779712,"internally_reserved":0,"allocated":1024405504,"data_stored":1023519808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1044040711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,9,6,8,1,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17891328,"num_objects":-4368,"num_object_clones":0,"num_object_copies":-8736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-22,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":-4368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4368,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005853"},"pg_stats":[{"pgid":"2.f","version":"43'14010","reported_seq":14220,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.417598+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:02.417598+0000","last_peered":"2024-09-11T01:21:02.417598+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:02.417598+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25509888,"num_objects":6228,"num_object_clones":0,"num_object_copies":12456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14010,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2173},{"shards":"0,2","objects":232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14087","reported_seq":14167,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.597596+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:00.597596+0000","last_peered":"2024-09-11T01:21:00.597596+0000","last_clean":"2024-09-11T01:21:00.597596+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:00.597596+0000","last_undegraded":"2024-09-11T01:21:00.597596+0000","last_fullsized":"2024-09-11T01:21:00.597596+0000","mapping_epoch":31,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26358726,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12185,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13768","reported_seq":13928,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.675129+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:00.675129+0000","last_peered":"2024-09-11T01:21:00.675129+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:00.675129+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25710616,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11829,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":261},{"shards":"2","objects":2311}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'13977","reported_seq":14022,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.653564+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:00.653564+0000","last_peered":"2024-09-11T01:21:00.653564+0000","last_clean":"2024-09-11T01:21:00.653564+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:00.653564+0000","last_undegraded":"2024-09-11T01:21:00.653564+0000","last_fullsized":"2024-09-11T01:21:00.653564+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26152960,"num_objects":6385,"num_object_clones":0,"num_object_copies":12770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13977,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'13979","reported_seq":14101,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.482085+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:02.482085+0000","last_peered":"2024-09-11T01:21:02.482085+0000","last_clean":"2024-09-11T01:21:02.482085+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:02.482085+0000","last_undegraded":"2024-09-11T01:21:02.482085+0000","last_fullsized":"2024-09-11T01:21:02.482085+0000","mapping_epoch":31,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25797091,"num_objects":6299,"num_object_clones":0,"num_object_copies":12598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10777,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'13981","reported_seq":14240,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.470156+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:02.470156+0000","last_peered":"2024-09-11T01:21:02.470156+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:02.470156+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25502179,"num_objects":6227,"num_object_clones":0,"num_object_copies":12454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4054,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10469,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2174},{"shards":"0,2","objects":239}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14431","reported_seq":14705,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.404412+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:02.404412+0000","last_peered":"2024-09-11T01:21:02.404412+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:02.404412+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26137542,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4213,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11184,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":70,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2170},{"shards":"0,2","objects":261}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14105","reported_seq":14225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.433120+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:02.433120+0000","last_peered":"2024-09-11T01:21:02.433120+0000","last_clean":"2024-09-11T01:21:02.433120+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:02.433120+0000","last_undegraded":"2024-09-11T01:21:02.433120+0000","last_fullsized":"2024-09-11T01:21:02.433120+0000","mapping_epoch":31,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25882624,"num_objects":6319,"num_object_clones":0,"num_object_copies":12638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11058,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'13979","reported_seq":22322,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.949551+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:20:59.949551+0000","last_peered":"2024-09-11T01:20:59.949551+0000","last_clean":"2024-09-11T01:20:59.949551+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:20:59.949551+0000","last_undegraded":"2024-09-11T01:20:59.949551+0000","last_fullsized":"2024-09-11T01:20:59.949551+0000","mapping_epoch":40,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25682403,"num_objects":6271,"num_object_clones":0,"num_object_copies":12542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11846,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14167","reported_seq":14426,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.427145+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:02.427145+0000","last_peered":"2024-09-11T01:21:02.427145+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:02.427145+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26324992,"num_objects":6427,"num_object_clones":0,"num_object_copies":12854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10531,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":65,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2387},{"shards":"0,2","objects":239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13796","reported_seq":14870,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.666507+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:00.666507+0000","last_peered":"2024-09-11T01:21:00.666507+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:00.666507+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"32'3700","ondisk_log_start":"32'3700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25981411,"num_objects":6344,"num_object_clones":0,"num_object_copies":12688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":6344,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11565,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":489,"num_bytes_recovered":1827299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":276},{"shards":"2","objects":2805}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14184","reported_seq":14330,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.555795+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:00.555795+0000","last_peered":"2024-09-11T01:21:00.555795+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:00.555795+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11821,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":277},{"shards":"2","objects":2227}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14047","reported_seq":22518,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.988931+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:20:59.988931+0000","last_peered":"2024-09-11T01:20:59.988931+0000","last_clean":"2024-09-11T01:20:59.988931+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:20:59.988931+0000","last_undegraded":"2024-09-11T01:20:59.988931+0000","last_fullsized":"2024-09-11T01:20:59.988931+0000","mapping_epoch":42,"log_start":"35'4000","ondisk_log_start":"35'4000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25825280,"num_objects":6305,"num_object_clones":0,"num_object_copies":12610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10737,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14257","reported_seq":14536,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.445457+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:02.445457+0000","last_peered":"2024-09-11T01:21:02.445457+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:02.445457+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"36'4200","ondisk_log_start":"36'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26169344,"num_objects":6389,"num_object_clones":0,"num_object_copies":12778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10650,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2269},{"shards":"0,2","objects":262}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14222","reported_seq":14506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.471552+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:02.471552+0000","last_peered":"2024-09-11T01:21:02.471552+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:02.471552+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4075,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10757,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2305},{"shards":"0,2","objects":228}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'13985","reported_seq":14216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.483730+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:02.483730+0000","last_peered":"2024-09-11T01:21:02.483730+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:02.483730+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25203654,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10642,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2049},{"shards":"0,2","objects":244}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":414487286,"num_objects":101203,"num_object_clones":0,"num_object_copies":202406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40054,"num_objects_misplaced":276,"num_objects_unfound":0,"num_objects_dirty":101203,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":184038,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9304,"num_bytes_recovered":35775430,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1021722624,"data_stored":1021569960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160975,"ondisk_log_size":160975,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639728,"kb_used_data":337312,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93732112,"statfs":{"total":96636764160,"available":95981682688,"internally_reserved":0,"allocated":345407488,"data_stored":345125660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790021,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":636200,"kb_used_data":293528,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93735640,"statfs":{"total":96636764160,"available":95985295360,"internally_reserved":0,"allocated":300572672,"data_stored":300250637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744104,"kb_used_data":369556,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93627736,"statfs":{"total":96636764160,"available":95874801664,"internally_reserved":0,"allocated":378425344,"data_stored":378143511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":383512234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,3,1,0,0,0,0,0,4],"upper_bound":16384},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":377200640,"data_stored":377164510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300077056,"data_stored":300000724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344444928,"data_stored":344404726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:09.137 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:09.694 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:09.695 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:09.724 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-11T01:21:08.564623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405754633,"num_objects":99073,"num_object_clones":0,"num_object_copies":198146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40035,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":99073,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":186172,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9444,"num_bytes_recovered":36266969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2020652,"kb_used_data":988536,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281094868,"statfs":{"total":289910292480,"available":287841144832,"internally_reserved":0,"allocated":1012260864,"data_stored":1011375168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1056820231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,10,4,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22876160,"num_objects":-5585,"num_object_clones":0,"num_object_copies":-11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":-5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5585,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006218"},"pg_stats":[{"pgid":"2.f","version":"43'14151","reported_seq":14361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.525254+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:07.525254+0000","last_peered":"2024-09-11T01:21:07.525254+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:07.525254+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24932352,"num_objects":6087,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14151,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2032},{"shards":"0,2","objects":91}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14218","reported_seq":14298,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.584322+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:05.584322+0000","last_peered":"2024-09-11T01:21:05.584322+0000","last_clean":"2024-09-11T01:21:05.584322+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:05.584322+0000","last_undegraded":"2024-09-11T01:21:05.584322+0000","last_fullsized":"2024-09-11T01:21:05.584322+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25822150,"num_objects":6306,"num_object_clones":0,"num_object_copies":12612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6306,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12316,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13886","reported_seq":14046,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.660158+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:05.660158+0000","last_peered":"2024-09-11T01:21:05.660158+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:05.660158+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227288,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11947,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":143},{"shards":"2","objects":2193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14102","reported_seq":14147,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.692557+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:05.692557+0000","last_peered":"2024-09-11T01:21:05.692557+0000","last_clean":"2024-09-11T01:21:05.692557+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:05.692557+0000","last_undegraded":"2024-09-11T01:21:05.692557+0000","last_fullsized":"2024-09-11T01:21:05.692557+0000","mapping_epoch":31,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25640960,"num_objects":6260,"num_object_clones":0,"num_object_copies":12520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14102,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14122","reported_seq":14244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.468564+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:07.468564+0000","last_peered":"2024-09-11T01:21:07.468564+0000","last_clean":"2024-09-11T01:21:07.468564+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:07.468564+0000","last_undegraded":"2024-09-11T01:21:07.468564+0000","last_fullsized":"2024-09-11T01:21:07.468564+0000","mapping_epoch":31,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25211363,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10920,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14122","reported_seq":14385,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.493658+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:07.493658+0000","last_peered":"2024-09-11T01:21:07.493658+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:07.493658+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924643,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10610,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2035},{"shards":"0,2","objects":98}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14577","reported_seq":14855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.470404+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:07.470404+0000","last_peered":"2024-09-11T01:21:07.470404+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:07.470404+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25539526,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11330,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14238","reported_seq":14358,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.457045+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:07.457045+0000","last_peered":"2024-09-11T01:21:07.457045+0000","last_clean":"2024-09-11T01:21:07.457045+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:07.457045+0000","last_undegraded":"2024-09-11T01:21:07.457045+0000","last_fullsized":"2024-09-11T01:21:07.457045+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25337856,"num_objects":6186,"num_object_clones":0,"num_object_copies":12372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11191,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14102","reported_seq":22444,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.067856+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:05.067856+0000","last_peered":"2024-09-11T01:21:05.067856+0000","last_clean":"2024-09-11T01:21:05.067856+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:05.067856+0000","last_undegraded":"2024-09-11T01:21:05.067856+0000","last_fullsized":"2024-09-11T01:21:05.067856+0000","mapping_epoch":40,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178595,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11969,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14290","reported_seq":14555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.432869+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:07.432869+0000","last_peered":"2024-09-11T01:21:07.432869+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:07.432869+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13930","reported_seq":15244,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.665546+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:05.665546+0000","last_peered":"2024-09-11T01:21:05.665546+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:05.665546+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432547,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11699,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":2318819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":156},{"shards":"2","objects":2791}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14322","reported_seq":14472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.623066+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:05.623066+0000","last_peered":"2024-09-11T01:21:05.623066+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:05.623066+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11959,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":139},{"shards":"2","objects":2091}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14185","reported_seq":22656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.026162+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:05.026162+0000","last_peered":"2024-09-11T01:21:05.026162+0000","last_clean":"2024-09-11T01:21:05.026162+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:05.026162+0000","last_undegraded":"2024-09-11T01:21:05.026162+0000","last_fullsized":"2024-09-11T01:21:05.026162+0000","mapping_epoch":42,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14403","reported_seq":14688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.452010+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:07.452010+0000","last_peered":"2024-09-11T01:21:07.452010+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:07.452010+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25571328,"num_objects":6243,"num_object_clones":0,"num_object_copies":12486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2126},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14345","reported_seq":14643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.483498+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:07.483498+0000","last_peered":"2024-09-11T01:21:07.483498+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:07.483498+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25620480,"num_objects":6255,"num_object_clones":0,"num_object_copies":12510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10880,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2189},{"shards":"0,2","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14114","reported_seq":14345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.494246+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:07.494246+0000","last_peered":"2024-09-11T01:21:07.494246+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:07.494246+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24675270,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10771,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1920},{"shards":"0,2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405754614,"num_objects":99071,"num_object_clones":0,"num_object_copies":198142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40035,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":99071,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":186170,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9443,"num_bytes_recovered":36266950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1009766400,"data_stored":1009613736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160607,"ondisk_log_size":160607,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639608,"kb_used_data":333032,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732232,"statfs":{"total":96636764160,"available":95981805568,"internally_reserved":0,"allocated":341024768,"data_stored":340742940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790022,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639276,"kb_used_data":292440,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93732564,"statfs":{"total":96636764160,"available":95982145536,"internally_reserved":0,"allocated":299458560,"data_stored":299136525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741768,"kb_used_data":363064,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93630072,"statfs":{"total":96636764160,"available":95877193728,"internally_reserved":0,"allocated":371777536,"data_stored":371495703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,3,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370667520,"data_stored":370631390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298950656,"data_stored":298874324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340148224,"data_stored":340108022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:09.726 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:10.294 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:10.294 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:10.318 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-11T01:21:08.564623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":405754633,"num_objects":99073,"num_object_clones":0,"num_object_copies":198146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40035,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":99073,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":186172,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9444,"num_bytes_recovered":36266969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160610,"ondisk_log_size":160610,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2020652,"kb_used_data":988536,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281094868,"statfs":{"total":289910292480,"available":287841144832,"internally_reserved":0,"allocated":1012260864,"data_stored":1011375168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1056820231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,10,4,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22876160,"num_objects":-5585,"num_object_clones":0,"num_object_copies":-11170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-39,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":-5585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5585,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006218"},"pg_stats":[{"pgid":"2.f","version":"43'14151","reported_seq":14361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.525254+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:07.525254+0000","last_peered":"2024-09-11T01:21:07.525254+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:07.525254+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24932352,"num_objects":6087,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14151,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2032},{"shards":"0,2","objects":91}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14218","reported_seq":14298,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.584322+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:05.584322+0000","last_peered":"2024-09-11T01:21:05.584322+0000","last_clean":"2024-09-11T01:21:05.584322+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:05.584322+0000","last_undegraded":"2024-09-11T01:21:05.584322+0000","last_fullsized":"2024-09-11T01:21:05.584322+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25822150,"num_objects":6306,"num_object_clones":0,"num_object_copies":12612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6306,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12316,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13886","reported_seq":14046,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.660158+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:05.660158+0000","last_peered":"2024-09-11T01:21:05.660158+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:05.660158+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227288,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11947,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":143},{"shards":"2","objects":2193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14102","reported_seq":14147,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.692557+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:05.692557+0000","last_peered":"2024-09-11T01:21:05.692557+0000","last_clean":"2024-09-11T01:21:05.692557+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:05.692557+0000","last_undegraded":"2024-09-11T01:21:05.692557+0000","last_fullsized":"2024-09-11T01:21:05.692557+0000","mapping_epoch":31,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25640960,"num_objects":6260,"num_object_clones":0,"num_object_copies":12520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14102,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14122","reported_seq":14244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.468564+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:07.468564+0000","last_peered":"2024-09-11T01:21:07.468564+0000","last_clean":"2024-09-11T01:21:07.468564+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:07.468564+0000","last_undegraded":"2024-09-11T01:21:07.468564+0000","last_fullsized":"2024-09-11T01:21:07.468564+0000","mapping_epoch":31,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25211363,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10920,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14122","reported_seq":14385,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.493658+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:07.493658+0000","last_peered":"2024-09-11T01:21:07.493658+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:07.493658+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924643,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10610,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2035},{"shards":"0,2","objects":98}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14577","reported_seq":14855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.470404+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:07.470404+0000","last_peered":"2024-09-11T01:21:07.470404+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:07.470404+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25539526,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11330,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14238","reported_seq":14358,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.457045+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:07.457045+0000","last_peered":"2024-09-11T01:21:07.457045+0000","last_clean":"2024-09-11T01:21:07.457045+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:07.457045+0000","last_undegraded":"2024-09-11T01:21:07.457045+0000","last_fullsized":"2024-09-11T01:21:07.457045+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25337856,"num_objects":6186,"num_object_clones":0,"num_object_copies":12372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11191,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14102","reported_seq":22444,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.067856+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:05.067856+0000","last_peered":"2024-09-11T01:21:05.067856+0000","last_clean":"2024-09-11T01:21:05.067856+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:05.067856+0000","last_undegraded":"2024-09-11T01:21:05.067856+0000","last_fullsized":"2024-09-11T01:21:05.067856+0000","mapping_epoch":40,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25178595,"num_objects":6148,"num_object_clones":0,"num_object_copies":12296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11969,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14290","reported_seq":14555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.432869+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:07.432869+0000","last_peered":"2024-09-11T01:21:07.432869+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:07.432869+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13930","reported_seq":15244,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.665546+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:05.665546+0000","last_peered":"2024-09-11T01:21:05.665546+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:05.665546+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432547,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11699,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":2318819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":156},{"shards":"2","objects":2791}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14322","reported_seq":14472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.623066+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:05.623066+0000","last_peered":"2024-09-11T01:21:05.623066+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:05.623066+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11959,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":139},{"shards":"2","objects":2091}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14185","reported_seq":22656,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.026162+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:05.026162+0000","last_peered":"2024-09-11T01:21:05.026162+0000","last_clean":"2024-09-11T01:21:05.026162+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:05.026162+0000","last_undegraded":"2024-09-11T01:21:05.026162+0000","last_fullsized":"2024-09-11T01:21:05.026162+0000","mapping_epoch":42,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14403","reported_seq":14688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.452010+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:07.452010+0000","last_peered":"2024-09-11T01:21:07.452010+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:07.452010+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25571328,"num_objects":6243,"num_object_clones":0,"num_object_copies":12486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2126},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14345","reported_seq":14643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.483498+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:07.483498+0000","last_peered":"2024-09-11T01:21:07.483498+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:07.483498+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25620480,"num_objects":6255,"num_object_clones":0,"num_object_copies":12510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10880,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2189},{"shards":"0,2","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14114","reported_seq":14345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.494246+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:07.494246+0000","last_peered":"2024-09-11T01:21:07.494246+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:07.494246+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24675270,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10771,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1920},{"shards":"0,2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":405754614,"num_objects":99071,"num_object_clones":0,"num_object_copies":198142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40035,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":99071,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":186170,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9443,"num_bytes_recovered":36266950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1009766400,"data_stored":1009613736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160607,"ondisk_log_size":160607,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639608,"kb_used_data":333032,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732232,"statfs":{"total":96636764160,"available":95981805568,"internally_reserved":0,"allocated":341024768,"data_stored":340742940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790022,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639276,"kb_used_data":292440,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93732564,"statfs":{"total":96636764160,"available":95982145536,"internally_reserved":0,"allocated":299458560,"data_stored":299136525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741768,"kb_used_data":363064,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93630072,"statfs":{"total":96636764160,"available":95877193728,"internally_reserved":0,"allocated":371777536,"data_stored":371495703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,3,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370667520,"data_stored":370631390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298950656,"data_stored":298874324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340148224,"data_stored":340108022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:10.319 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:10.319 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14151", 'reported_seq': 14361, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.525254+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:07.525254+0000', 'last_peered': '2024-09-11T01:21:07.525254+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:07.525254+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "36'4100", 'ondisk_log_start': "36'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24932352, 'num_objects': 6087, 'num_object_clones': 0, 'num_object_copies': 12174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4055, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6087, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14151, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2032}, {'shards': '0,2', 'objects': 91}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.319 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:10.319 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'13886", 'reported_seq': 14046, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:05.660158+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:05.660158+0000', 'last_peered': '2024-09-11T01:21:05.660158+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:05.660158+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "35'3800", 'ondisk_log_start': "35'3800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25227288, 'num_objects': 6160, 'num_object_clones': 0, 'num_object_copies': 12320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3968, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6160, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11947, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 143}, {'shards': '2', 'objects': 2193}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14122", 'reported_seq': 14385, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.493658+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:07.493658+0000', 'last_peered': '2024-09-11T01:21:07.493658+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:07.493658+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "36'4100", 'ondisk_log_start': "36'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24924643, 'num_objects': 6086, 'num_object_clones': 0, 'num_object_copies': 12172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4052, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6086, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10610, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 64, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2035}, {'shards': '0,2', 'objects': 98}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'14577", 'reported_seq': 14855, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.470404+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:07.470404+0000', 'last_peered': '2024-09-11T01:21:07.470404+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:07.470404+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25539526, 'num_objects': 6237, 'num_object_clones': 0, 'num_object_copies': 12474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4211, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6237, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11330, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 72, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2026}, {'shards': '0,2', 'objects': 115}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14290", 'reported_seq': 14555, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.432869+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:07.432869+0000', 'last_peered': '2024-09-11T01:21:07.432869+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:07.432869+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25821184, 'num_objects': 6304, 'num_object_clones': 0, 'num_object_copies': 12608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4039, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10654, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 68, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2267}, {'shards': '0,2', 'objects': 116}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'13930", 'reported_seq': 15244, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:05.665546+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:05.665546+0000', 'last_peered': '2024-09-11T01:21:05.665546+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:05.665546+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "35'3900", 'ondisk_log_start': "35'3900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25432547, 'num_objects': 6210, 'num_object_clones': 0, 'num_object_copies': 12420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 156, 'num_objects_unfound': 0, 'num_objects_dirty': 6210, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11699, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 609, 'num_bytes_recovered': 2318819, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 156}, {'shards': '2', 'objects': 2791}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:10.320 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'14322", 'reported_seq': 14472, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:05.623066+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:05.623066+0000', 'last_peered': '2024-09-11T01:21:05.623066+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:05.623066+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25559040, 'num_objects': 6240, 'num_object_clones': 0, 'num_object_copies': 12480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4151, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11959, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 28, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 139}, {'shards': '2', 'objects': 2091}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:10.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:10.321 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'14403", 'reported_seq': 14688, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.452010+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:07.452010+0000', 'last_peered': '2024-09-11T01:21:07.452010+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:07.452010+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25571328, 'num_objects': 6243, 'num_object_clones': 0, 'num_object_copies': 12486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6243, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10796, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2126}, {'shards': '0,2', 'objects': 116}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:10.321 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14345", 'reported_seq': 14643, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.483498+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:07.483498+0000', 'last_peered': '2024-09-11T01:21:07.483498+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:07.483498+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25620480, 'num_objects': 6255, 'num_object_clones': 0, 'num_object_copies': 12510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4068, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10880, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 82, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2189}, {'shards': '0,2', 'objects': 105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.321 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:10.321 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14114", 'reported_seq': 14345, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.494246+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:07.494246+0000', 'last_peered': '2024-09-11T01:21:07.494246+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:07.494246+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "35'4100", 'ondisk_log_start': "35'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24675270, 'num_objects': 6026, 'num_object_clones': 0, 'num_object_copies': 12052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4108, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6026, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10771, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 48, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1920}, {'shards': '0,2', 'objects': 115}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:10.321 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:11.002 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:11.003 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40035/197636 objects degraded (20.257%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":471,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98818,"data_bytes":404710153,"bytes_used":2059657216,"bytes_avail":287850635264,"bytes_total":289910292480,"degraded_objects":40035,"degraded_total":197636,"degraded_ratio":0.20256936995284261,"misplaced_objects":156,"misplaced_total":197636,"misplaced_ratio":0.00078932987917181073,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":75056,"recovering_keys_per_sec":0,"num_objects_recovered":250,"num_bytes_recovered":901120,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:11.032 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:11.033 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:11.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:11.597 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:11.623 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T01:21:10.565106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404710153,"num_objects":98818,"num_object_clones":0,"num_object_copies":197636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40035,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":98818,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":186427,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9444,"num_bytes_recovered":36266969,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160565,"ondisk_log_size":160565,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2011384,"kb_used_data":987584,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281104136,"statfs":{"total":289910292480,"available":287850635264,"internally_reserved":0,"allocated":1011286016,"data_stored":1010400320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1048300551},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,10,2,4,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18911232,"num_objects":-4617,"num_object_clones":0,"num_object_copies":-9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-30,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":-4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005890"},"pg_stats":[{"pgid":"2.f","version":"43'14151","reported_seq":14361,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.525254+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:07.525254+0000","last_peered":"2024-09-11T01:21:07.525254+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:07.525254+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24932352,"num_objects":6087,"num_object_clones":0,"num_object_copies":12174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14151,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2032},{"shards":"0,2","objects":91}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14218","reported_seq":14298,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.584322+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:05.584322+0000","last_peered":"2024-09-11T01:21:05.584322+0000","last_clean":"2024-09-11T01:21:05.584322+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:05.584322+0000","last_undegraded":"2024-09-11T01:21:05.584322+0000","last_fullsized":"2024-09-11T01:21:05.584322+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25822150,"num_objects":6306,"num_object_clones":0,"num_object_copies":12612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6306,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12316,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'13886","reported_seq":14046,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.660158+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:05.660158+0000","last_peered":"2024-09-11T01:21:05.660158+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:05.660158+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"35'3800","ondisk_log_start":"35'3800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227288,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3968,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11947,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":143},{"shards":"2","objects":2193}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14102","reported_seq":14147,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.692557+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:05.692557+0000","last_peered":"2024-09-11T01:21:05.692557+0000","last_clean":"2024-09-11T01:21:05.692557+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:05.692557+0000","last_undegraded":"2024-09-11T01:21:05.692557+0000","last_fullsized":"2024-09-11T01:21:05.692557+0000","mapping_epoch":31,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25640960,"num_objects":6260,"num_object_clones":0,"num_object_copies":12520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14102,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14122","reported_seq":14244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.468564+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:07.468564+0000","last_peered":"2024-09-11T01:21:07.468564+0000","last_clean":"2024-09-11T01:21:07.468564+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:07.468564+0000","last_undegraded":"2024-09-11T01:21:07.468564+0000","last_fullsized":"2024-09-11T01:21:07.468564+0000","mapping_epoch":31,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25211363,"num_objects":6156,"num_object_clones":0,"num_object_copies":12312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10920,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14122","reported_seq":14385,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.493658+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:07.493658+0000","last_peered":"2024-09-11T01:21:07.493658+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:07.493658+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"36'4100","ondisk_log_start":"36'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924643,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10610,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2035},{"shards":"0,2","objects":98}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14577","reported_seq":14855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.470404+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:07.470404+0000","last_peered":"2024-09-11T01:21:07.470404+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:07.470404+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25539526,"num_objects":6237,"num_object_clones":0,"num_object_copies":12474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6237,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11330,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":72,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2026},{"shards":"0,2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14238","reported_seq":14358,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.457045+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:07.457045+0000","last_peered":"2024-09-11T01:21:07.457045+0000","last_clean":"2024-09-11T01:21:07.457045+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:07.457045+0000","last_undegraded":"2024-09-11T01:21:07.457045+0000","last_fullsized":"2024-09-11T01:21:07.457045+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25337856,"num_objects":6186,"num_object_clones":0,"num_object_copies":12372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11191,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14225","reported_seq":22568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.036407+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:10.036407+0000","last_peered":"2024-09-11T01:21:10.036407+0000","last_clean":"2024-09-11T01:21:10.036407+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:10.036407+0000","last_undegraded":"2024-09-11T01:21:10.036407+0000","last_fullsized":"2024-09-11T01:21:10.036407+0000","mapping_epoch":40,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674787,"num_objects":6025,"num_object_clones":0,"num_object_copies":12050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12092,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14290","reported_seq":14555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.432869+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:07.432869+0000","last_peered":"2024-09-11T01:21:07.432869+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:07.432869+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10654,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":68,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2267},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'13930","reported_seq":15244,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.665546+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:05.665546+0000","last_peered":"2024-09-11T01:21:05.665546+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:05.665546+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"35'3900","ondisk_log_start":"35'3900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432547,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11699,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":2318819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":156},{"shards":"2","objects":2791}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14322","reported_seq":14472,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.623066+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:05.623066+0000","last_peered":"2024-09-11T01:21:05.623066+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:05.623066+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25559040,"num_objects":6240,"num_object_clones":0,"num_object_copies":12480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11959,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":139},{"shards":"2","objects":2091}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14317","reported_seq":22788,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.128061+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:10.128061+0000","last_peered":"2024-09-11T01:21:10.128061+0000","last_clean":"2024-09-11T01:21:10.128061+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:10.128061+0000","last_undegraded":"2024-09-11T01:21:10.128061+0000","last_fullsized":"2024-09-11T01:21:10.128061+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24719360,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14403","reported_seq":14688,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.452010+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:07.452010+0000","last_peered":"2024-09-11T01:21:07.452010+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:07.452010+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25571328,"num_objects":6243,"num_object_clones":0,"num_object_copies":12486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10796,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2126},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14345","reported_seq":14643,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.483498+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:07.483498+0000","last_peered":"2024-09-11T01:21:07.483498+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:07.483498+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25620480,"num_objects":6255,"num_object_clones":0,"num_object_copies":12510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10880,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":82,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2189},{"shards":"0,2","objects":105}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14114","reported_seq":14345,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.494246+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:07.494246+0000","last_peered":"2024-09-11T01:21:07.494246+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:07.494246+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"35'4100","ondisk_log_start":"35'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24675270,"num_objects":6026,"num_object_clones":0,"num_object_copies":12052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10771,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1920},{"shards":"0,2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404710134,"num_objects":98816,"num_object_clones":0,"num_object_copies":197632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40035,"num_objects_misplaced":156,"num_objects_unfound":0,"num_objects_dirty":98816,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":186425,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9443,"num_bytes_recovered":36266950,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1008750592,"data_stored":1008597928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160562,"ondisk_log_size":160562,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639608,"kb_used_data":333032,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93732232,"statfs":{"total":96636764160,"available":95981805568,"internally_reserved":0,"allocated":341024768,"data_stored":340742940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790023,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630008,"kb_used_data":291488,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93741832,"statfs":{"total":96636764160,"available":95991635968,"internally_reserved":0,"allocated":298483712,"data_stored":298161677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":741768,"kb_used_data":363064,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93630072,"statfs":{"total":96636764160,"available":95877193728,"internally_reserved":0,"allocated":371777536,"data_stored":371495703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":387772074},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,3,0,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":370667520,"data_stored":370631390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297934848,"data_stored":297858516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340148224,"data_stored":340108022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:14.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:15.217 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:15.220 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:15.245 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T01:21:14.566501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":397058825,"num_objects":96950,"num_object_clones":0,"num_object_copies":193900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40011,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":96950,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":188295,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9614,"num_bytes_recovered":36864985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009280,"kb_used_data":977160,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281106240,"statfs":{"total":289910292480,"available":287852789760,"internally_reserved":0,"allocated":1000611840,"data_stored":999726144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1056820231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,8,5,3,0,0,3,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17428480,"num_objects":-4255,"num_object_clones":0,"num_object_copies":-8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4255,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004531"},"pg_stats":[{"pgid":"2.f","version":"43'14267","reported_seq":14489,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.474670+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:12.474670+0000","last_peered":"2024-09-11T01:21:12.474670+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:12.474670+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24457216,"num_objects":5971,"num_object_clones":0,"num_object_copies":11942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4049,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14334","reported_seq":14415,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.632552+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:10.632552+0000","last_peered":"2024-09-11T01:21:10.632552+0000","last_clean":"2024-09-11T01:21:10.632552+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:10.632552+0000","last_undegraded":"2024-09-11T01:21:10.632552+0000","last_fullsized":"2024-09-11T01:21:10.632552+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25347014,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12432,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14011","reported_seq":14175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.590028+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:10.590028+0000","last_peered":"2024-09-11T01:21:10.590028+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:10.590028+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24715288,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3966,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12072,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18},{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14231","reported_seq":14277,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.698284+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:10.698284+0000","last_peered":"2024-09-11T01:21:10.698284+0000","last_clean":"2024-09-11T01:21:10.698284+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:10.698284+0000","last_undegraded":"2024-09-11T01:21:10.698284+0000","last_fullsized":"2024-09-11T01:21:10.698284+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112576,"num_objects":6131,"num_object_clones":0,"num_object_copies":12262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14231,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14264","reported_seq":14386,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.501928+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:12.501928+0000","last_peered":"2024-09-11T01:21:12.501928+0000","last_clean":"2024-09-11T01:21:12.501928+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:12.501928+0000","last_undegraded":"2024-09-11T01:21:12.501928+0000","last_fullsized":"2024-09-11T01:21:12.501928+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24629731,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11062,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14247","reported_seq":14514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.440645+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:12.440645+0000","last_peered":"2024-09-11T01:21:12.440645+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:12.440645+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412643,"num_objects":5961,"num_object_clones":0,"num_object_copies":11922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5961,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10735,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1912},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14720","reported_seq":15008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.422718+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:12.422718+0000","last_peered":"2024-09-11T01:21:12.422718+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:12.422718+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24953798,"num_objects":6094,"num_object_clones":0,"num_object_copies":12188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6094,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11473,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14360","reported_seq":14480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.515254+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:12.515254+0000","last_peered":"2024-09-11T01:21:12.515254+0000","last_clean":"2024-09-11T01:21:12.515254+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:12.515254+0000","last_undegraded":"2024-09-11T01:21:12.515254+0000","last_fullsized":"2024-09-11T01:21:12.515254+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14225","reported_seq":22568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.036407+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:10.036407+0000","last_peered":"2024-09-11T01:21:10.036407+0000","last_clean":"2024-09-11T01:21:10.036407+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:10.036407+0000","last_undegraded":"2024-09-11T01:21:10.036407+0000","last_fullsized":"2024-09-11T01:21:10.036407+0000","mapping_epoch":40,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674787,"num_objects":6025,"num_object_clones":0,"num_object_copies":12050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12092,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14439","reported_seq":14716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.491522+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:12.491522+0000","last_peered":"2024-09-11T01:21:12.491522+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:12.491522+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25210880,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14075","reported_seq":15681,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.688504+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:10.688504+0000","last_peered":"2024-09-11T01:21:10.688504+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:10.688504+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838627,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11844,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":755,"num_bytes_recovered":2916835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":10},{"shards":"2","objects":2792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14443","reported_seq":14595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.482331+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:10.482331+0000","last_peered":"2024-09-11T01:21:10.482331+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:10.482331+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12080,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18},{"shards":"2","objects":1971}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14317","reported_seq":22788,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.128061+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:10.128061+0000","last_peered":"2024-09-11T01:21:10.128061+0000","last_clean":"2024-09-11T01:21:10.128061+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:10.128061+0000","last_undegraded":"2024-09-11T01:21:10.128061+0000","last_fullsized":"2024-09-11T01:21:10.128061+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24719360,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14550","reported_seq":14835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.506427+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:12.506427+0000","last_peered":"2024-09-11T01:21:12.506427+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:12.506427+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14479","reported_seq":14779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.517033+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:12.517033+0000","last_peered":"2024-09-11T01:21:12.517033+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:12.517033+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25071616,"num_objects":6121,"num_object_clones":0,"num_object_copies":12242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14268","reported_seq":14500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.507545+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:12.507545+0000","last_peered":"2024-09-11T01:21:12.507545+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:12.507545+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24044486,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10925,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1767},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":397058806,"num_objects":96948,"num_object_clones":0,"num_object_copies":193896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40011,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":96948,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":188293,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9613,"num_bytes_recovered":36864966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":997847040,"data_stored":997694376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639668,"kb_used_data":328932,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93732172,"statfs":{"total":96636764160,"available":95981744128,"internally_reserved":0,"allocated":336826368,"data_stored":336544540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,4,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790023,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630008,"kb_used_data":291488,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93741832,"statfs":{"total":96636764160,"available":95991635968,"internally_reserved":0,"allocated":298483712,"data_stored":298161677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739604,"kb_used_data":356740,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93632236,"statfs":{"total":96636764160,"available":95879409664,"internally_reserved":0,"allocated":365301760,"data_stored":365019927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,1,2,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364011520,"data_stored":363975390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297934848,"data_stored":297858516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335900672,"data_stored":335860470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:15.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:15.823 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:15.823 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:15.847 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T01:21:14.566501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":397058825,"num_objects":96950,"num_object_clones":0,"num_object_copies":193900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40011,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":96950,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":188295,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9614,"num_bytes_recovered":36864985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160733,"ondisk_log_size":160733,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009280,"kb_used_data":977160,"kb_used_omap":12,"kb_used_meta":1032051,"kb_avail":281106240,"statfs":{"total":289910292480,"available":287852789760,"internally_reserved":0,"allocated":1000611840,"data_stored":999726144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1056820231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,8,5,3,0,0,3,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17428480,"num_objects":-4255,"num_object_clones":0,"num_object_copies":-8510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-43,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":-4255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4255,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004531"},"pg_stats":[{"pgid":"2.f","version":"43'14267","reported_seq":14489,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.474670+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:12.474670+0000","last_peered":"2024-09-11T01:21:12.474670+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:12.474670+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24457216,"num_objects":5971,"num_object_clones":0,"num_object_copies":11942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4049,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14334","reported_seq":14415,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.632552+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:10.632552+0000","last_peered":"2024-09-11T01:21:10.632552+0000","last_clean":"2024-09-11T01:21:10.632552+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:10.632552+0000","last_undegraded":"2024-09-11T01:21:10.632552+0000","last_fullsized":"2024-09-11T01:21:10.632552+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25347014,"num_objects":6190,"num_object_clones":0,"num_object_copies":12380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6190,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12432,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14011","reported_seq":14175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.590028+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:10.590028+0000","last_peered":"2024-09-11T01:21:10.590028+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:10.590028+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24715288,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3966,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12072,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18},{"shards":"2","objects":2070}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14231","reported_seq":14277,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.698284+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:10.698284+0000","last_peered":"2024-09-11T01:21:10.698284+0000","last_clean":"2024-09-11T01:21:10.698284+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:10.698284+0000","last_undegraded":"2024-09-11T01:21:10.698284+0000","last_fullsized":"2024-09-11T01:21:10.698284+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112576,"num_objects":6131,"num_object_clones":0,"num_object_copies":12262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14231,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14264","reported_seq":14386,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.501928+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:12.501928+0000","last_peered":"2024-09-11T01:21:12.501928+0000","last_clean":"2024-09-11T01:21:12.501928+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:12.501928+0000","last_undegraded":"2024-09-11T01:21:12.501928+0000","last_fullsized":"2024-09-11T01:21:12.501928+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24629731,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11062,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14247","reported_seq":14514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.440645+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:12.440645+0000","last_peered":"2024-09-11T01:21:12.440645+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:12.440645+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412643,"num_objects":5961,"num_object_clones":0,"num_object_copies":11922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5961,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10735,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1912},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14720","reported_seq":15008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.422718+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:12.422718+0000","last_peered":"2024-09-11T01:21:12.422718+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:12.422718+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24953798,"num_objects":6094,"num_object_clones":0,"num_object_copies":12188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6094,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11473,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14360","reported_seq":14480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.515254+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:12.515254+0000","last_peered":"2024-09-11T01:21:12.515254+0000","last_clean":"2024-09-11T01:21:12.515254+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:12.515254+0000","last_undegraded":"2024-09-11T01:21:12.515254+0000","last_fullsized":"2024-09-11T01:21:12.515254+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14225","reported_seq":22568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.036407+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:10.036407+0000","last_peered":"2024-09-11T01:21:10.036407+0000","last_clean":"2024-09-11T01:21:10.036407+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:10.036407+0000","last_undegraded":"2024-09-11T01:21:10.036407+0000","last_fullsized":"2024-09-11T01:21:10.036407+0000","mapping_epoch":40,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674787,"num_objects":6025,"num_object_clones":0,"num_object_copies":12050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6025,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12092,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14439","reported_seq":14716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.491522+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:12.491522+0000","last_peered":"2024-09-11T01:21:12.491522+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:12.491522+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25210880,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14075","reported_seq":15681,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.688504+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:10.688504+0000","last_peered":"2024-09-11T01:21:10.688504+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:10.688504+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"36'4000","ondisk_log_start":"36'4000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838627,"num_objects":6065,"num_object_clones":0,"num_object_copies":12130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3263,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":6065,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11844,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":755,"num_bytes_recovered":2916835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":10},{"shards":"2","objects":2792}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14443","reported_seq":14595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.482331+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:10.482331+0000","last_peered":"2024-09-11T01:21:10.482331+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:10.482331+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4150,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12080,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":18},{"shards":"2","objects":1971}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14317","reported_seq":22788,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.128061+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:10.128061+0000","last_peered":"2024-09-11T01:21:10.128061+0000","last_clean":"2024-09-11T01:21:10.128061+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:10.128061+0000","last_undegraded":"2024-09-11T01:21:10.128061+0000","last_fullsized":"2024-09-11T01:21:10.128061+0000","mapping_epoch":42,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24719360,"num_objects":6035,"num_object_clones":0,"num_object_copies":12070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11007,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14550","reported_seq":14835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.506427+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:12.506427+0000","last_peered":"2024-09-11T01:21:12.506427+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:12.506427+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14479","reported_seq":14779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.517033+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:12.517033+0000","last_peered":"2024-09-11T01:21:12.517033+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:12.517033+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25071616,"num_objects":6121,"num_object_clones":0,"num_object_copies":12242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14268","reported_seq":14500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.507545+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:12.507545+0000","last_peered":"2024-09-11T01:21:12.507545+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:12.507545+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24044486,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10925,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1767},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":397058806,"num_objects":96948,"num_object_clones":0,"num_object_copies":193896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40011,"num_objects_misplaced":10,"num_objects_unfound":0,"num_objects_dirty":96948,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":188293,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9613,"num_bytes_recovered":36864966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":997847040,"data_stored":997694376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639668,"kb_used_data":328932,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93732172,"statfs":{"total":96636764160,"available":95981744128,"internally_reserved":0,"allocated":336826368,"data_stored":336544540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,4,1,0,0,0,0,0,3],"upper_bound":8192},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790023,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630008,"kb_used_data":291488,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93741832,"statfs":{"total":96636764160,"available":95991635968,"internally_reserved":0,"allocated":298483712,"data_stored":298161677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739604,"kb_used_data":356740,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93632236,"statfs":{"total":96636764160,"available":95879409664,"internally_reserved":0,"allocated":365301760,"data_stored":365019927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,1,2,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364011520,"data_stored":363975390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":297934848,"data_stored":297858516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335900672,"data_stored":335860470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:15.849 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:15.849 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14267", 'reported_seq': 14489, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.474670+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:12.474670+0000', 'last_peered': '2024-09-11T01:21:12.474670+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:12.474670+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24457216, 'num_objects': 5971, 'num_object_clones': 0, 'num_object_copies': 11942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4049, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5971, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14267, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 74, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1922}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.849 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:15.849 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'14011", 'reported_seq': 14175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:10.590028+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:10.590028+0000', 'last_peered': '2024-09-11T01:21:10.590028+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:10.590028+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "36'4000", 'ondisk_log_start': "36'4000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24715288, 'num_objects': 6035, 'num_object_clones': 0, 'num_object_copies': 12070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3966, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6035, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12072, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 41, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 18}, {'shards': '2', 'objects': 2070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:15.849 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:15.849 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14247", 'reported_seq': 14514, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.440645+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:12.440645+0000', 'last_peered': '2024-09-11T01:21:12.440645+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:12.440645+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24412643, 'num_objects': 5961, 'num_object_clones': 0, 'num_object_copies': 11922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5961, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10735, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1912}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'14720", 'reported_seq': 15008, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.422718+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:12.422718+0000', 'last_peered': '2024-09-11T01:21:12.422718+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:12.422718+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'4700", 'ondisk_log_start': "39'4700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24953798, 'num_objects': 6094, 'num_object_clones': 0, 'num_object_copies': 12188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6094, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11473, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1888}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14439", 'reported_seq': 14716, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.491522+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:12.491522+0000', 'last_peered': '2024-09-11T01:21:12.491522+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:12.491522+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25210880, 'num_objects': 6155, 'num_object_clones': 0, 'num_object_copies': 12310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4033, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6155, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10803, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 74, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2124}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'14075", 'reported_seq': 15681, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:10.688504+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:10.688504+0000', 'last_peered': '2024-09-11T01:21:10.688504+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:10.688504+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "36'4000", 'ondisk_log_start': "36'4000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24838627, 'num_objects': 6065, 'num_object_clones': 0, 'num_object_copies': 12130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3263, 'num_objects_misplaced': 10, 'num_objects_unfound': 0, 'num_objects_dirty': 6065, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11844, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 755, 'num_bytes_recovered': 2916835, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 10}, {'shards': '2', 'objects': 2792}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'14443", 'reported_seq': 14595, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:10.482331+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:10.482331+0000', 'last_peered': '2024-09-11T01:21:10.482331+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:10.482331+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25063424, 'num_objects': 6119, 'num_object_clones': 0, 'num_object_copies': 12238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4150, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12080, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 29, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 18}, {'shards': '2', 'objects': 1971}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'14550", 'reported_seq': 14835, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.506427+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:12.506427+0000', 'last_peered': '2024-09-11T01:21:12.506427+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:12.506427+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24969216, 'num_objects': 6096, 'num_object_clones': 0, 'num_object_copies': 12192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10943, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1979}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:15.850 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14479", 'reported_seq': 14779, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.517033+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:12.517033+0000', 'last_peered': '2024-09-11T01:21:12.517033+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:12.517033+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25071616, 'num_objects': 6121, 'num_object_clones': 0, 'num_object_copies': 12242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6121, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11014, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2056}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.851 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:15.851 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14268", 'reported_seq': 14500, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.507545+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:12.507545+0000', 'last_peered': '2024-09-11T01:21:12.507545+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:12.507545+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24044486, 'num_objects': 5872, 'num_object_clones': 0, 'num_object_copies': 11744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4107, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5872, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10925, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 49, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1767}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:15.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:16.556 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:16.556 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40011/193900 objects degraded (20.635%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":477,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":96950,"data_bytes":397058825,"bytes_used":2057502720,"bytes_avail":287852789760,"bytes_total":289910292480,"degraded_objects":40011,"degraded_total":193900,"degraded_ratio":0.20634863331614234,"misplaced_objects":10,"misplaced_total":193900,"misplaced_ratio":5.1572975760701392e-05,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":90760,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":1089536,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":354},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:16.578 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:16.578 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:17.141 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:17.143 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:17.162 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-11T01:21:16.569618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":392909577,"num_objects":95937,"num_object_clones":0,"num_object_copies":191874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95937,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":189308,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9722,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160946,"ondisk_log_size":160946,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1991612,"kb_used_data":967820,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281123908,"statfs":{"total":289910292480,"available":287870881792,"internally_reserved":0,"allocated":991047680,"data_stored":990161984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1048300551},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,10,6,8,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21577728,"num_objects":-5268,"num_object_clones":0,"num_object_copies":-10536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-141,"num_objects_misplaced":-276,"num_objects_unfound":0,"num_objects_dirty":-5268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5268,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":1130496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006672"},"pg_stats":[{"pgid":"2.f","version":"43'14267","reported_seq":14489,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.474670+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:12.474670+0000","last_peered":"2024-09-11T01:21:12.474670+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:12.474670+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24457216,"num_objects":5971,"num_object_clones":0,"num_object_copies":11942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4049,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14267,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14487","reported_seq":14568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.660442+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:15.660442+0000","last_peered":"2024-09-11T01:21:15.660442+0000","last_clean":"2024-09-11T01:21:15.660442+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:15.660442+0000","last_undegraded":"2024-09-11T01:21:15.660442+0000","last_fullsized":"2024-09-11T01:21:15.660442+0000","mapping_epoch":31,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24720326,"num_objects":6037,"num_object_clones":0,"num_object_copies":12074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12585,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14149","reported_seq":14317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.639553+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:15.639553+0000","last_peered":"2024-09-11T01:21:15.639553+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:15.639553+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150040,"num_objects":5897,"num_object_clones":0,"num_object_copies":11794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12210,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14378","reported_seq":14424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.594277+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:15.594277+0000","last_peered":"2024-09-11T01:21:15.594277+0000","last_clean":"2024-09-11T01:21:15.594277+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:15.594277+0000","last_undegraded":"2024-09-11T01:21:15.594277+0000","last_fullsized":"2024-09-11T01:21:15.594277+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14378,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14264","reported_seq":14386,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.501928+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:12.501928+0000","last_peered":"2024-09-11T01:21:12.501928+0000","last_clean":"2024-09-11T01:21:12.501928+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:12.501928+0000","last_undegraded":"2024-09-11T01:21:12.501928+0000","last_fullsized":"2024-09-11T01:21:12.501928+0000","mapping_epoch":31,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24629731,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11062,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14247","reported_seq":14514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.440645+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:12.440645+0000","last_peered":"2024-09-11T01:21:12.440645+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:12.440645+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412643,"num_objects":5961,"num_object_clones":0,"num_object_copies":11922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5961,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10735,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1912},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14720","reported_seq":15008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.422718+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:12.422718+0000","last_peered":"2024-09-11T01:21:12.422718+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:12.422718+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24953798,"num_objects":6094,"num_object_clones":0,"num_object_copies":12188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6094,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11473,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1888},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14360","reported_seq":14480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.515254+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:12.515254+0000","last_peered":"2024-09-11T01:21:12.515254+0000","last_clean":"2024-09-11T01:21:12.515254+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:12.515254+0000","last_undegraded":"2024-09-11T01:21:12.515254+0000","last_fullsized":"2024-09-11T01:21:12.515254+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24838144,"num_objects":6064,"num_object_clones":0,"num_object_copies":12128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11313,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.679962+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.679962+0000","last_peered":"2024-09-11T01:20:45.679962+0000","last_clean":"2024-09-11T01:20:45.679962+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.679962+0000","last_undegraded":"2024-09-11T01:20:45.679962+0000","last_fullsized":"2024-09-11T01:20:45.679962+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14357","reported_seq":22700,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.251626+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:15.251626+0000","last_peered":"2024-09-11T01:21:15.251626+0000","last_clean":"2024-09-11T01:21:15.251626+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:15.251626+0000","last_undegraded":"2024-09-11T01:21:15.251626+0000","last_fullsized":"2024-09-11T01:21:15.251626+0000","mapping_epoch":40,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24134115,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12224,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14439","reported_seq":14716,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.491522+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:12.491522+0000","last_peered":"2024-09-11T01:21:12.491522+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:12.491522+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25210880,"num_objects":6155,"num_object_clones":0,"num_object_copies":12310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10803,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":74,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2124}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14228","reported_seq":16044,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.707509+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:15.707509+0000","last_peered":"2024-09-11T01:21:15.707509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:15.707509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24211939,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11997,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14588","reported_seq":14742,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.724270+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:15.724270+0000","last_peered":"2024-09-11T01:21:15.724270+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:15.724270+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24469504,"num_objects":5974,"num_object_clones":0,"num_object_copies":11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12225,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14462","reported_seq":22933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.228055+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:15.228055+0000","last_peered":"2024-09-11T01:21:15.228055+0000","last_clean":"2024-09-11T01:21:15.228055+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:15.228055+0000","last_undegraded":"2024-09-11T01:21:15.228055+0000","last_fullsized":"2024-09-11T01:21:15.228055+0000","mapping_epoch":42,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24125440,"num_objects":5890,"num_object_clones":0,"num_object_copies":11780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11152,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14550","reported_seq":14835,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.506427+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:12.506427+0000","last_peered":"2024-09-11T01:21:12.506427+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:12.506427+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24969216,"num_objects":6096,"num_object_clones":0,"num_object_copies":12192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14479","reported_seq":14779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.517033+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:12.517033+0000","last_peered":"2024-09-11T01:21:12.517033+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:12.517033+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25071616,"num_objects":6121,"num_object_clones":0,"num_object_copies":12242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11014,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14268","reported_seq":14500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.507545+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:12.507545+0000","last_peered":"2024-09-11T01:21:12.507545+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:12.507545+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24044486,"num_objects":5872,"num_object_clones":0,"num_object_copies":11744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5872,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10925,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1767},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":392909558,"num_objects":95935,"num_object_clones":0,"num_object_copies":191870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39913,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95935,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":189306,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9721,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":987275264,"data_stored":987122600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160943,"ondisk_log_size":160943,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622424,"kb_used_data":324168,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93749416,"statfs":{"total":96636764160,"available":95999401984,"internally_reserved":0,"allocated":331948032,"data_stored":331666204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790024,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629584,"kb_used_data":286912,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93742256,"statfs":{"total":96636764160,"available":95992070144,"internally_reserved":0,"allocated":293797888,"data_stored":293475853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739604,"kb_used_data":356740,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93632236,"statfs":{"total":96636764160,"available":95879409664,"internally_reserved":0,"allocated":365301760,"data_stored":365019927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":392031914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,1,2,0,0,3],"upper_bound":1024},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364011520,"data_stored":363975390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292245504,"data_stored":292169172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331018240,"data_stored":330978038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:20.163 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:20.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:20.753 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:20.772 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T01:21:20.571112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386474761,"num_objects":94366,"num_object_clones":0,"num_object_copies":188732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94366,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":190879,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9727,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1991680,"kb_used_data":953988,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281123840,"statfs":{"total":289910292480,"available":287870812160,"internally_reserved":0,"allocated":976883712,"data_stored":975998016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1062521863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,6,8,7,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19279872,"num_objects":-4707,"num_object_clones":0,"num_object_copies":-9414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-127,"num_objects_misplaced":-156,"num_objects_unfound":0,"num_objects_dirty":-4707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":638976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006489"},"pg_stats":[{"pgid":"2.f","version":"43'14396","reported_seq":14617,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.469121+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:17.469121+0000","last_peered":"2024-09-11T01:21:17.469121+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:17.469121+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14396,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1794}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14487","reported_seq":14568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.660442+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:15.660442+0000","last_peered":"2024-09-11T01:21:15.660442+0000","last_clean":"2024-09-11T01:21:15.660442+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:15.660442+0000","last_undegraded":"2024-09-11T01:21:15.660442+0000","last_fullsized":"2024-09-11T01:21:15.660442+0000","mapping_epoch":31,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24720326,"num_objects":6037,"num_object_clones":0,"num_object_copies":12074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12585,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14149","reported_seq":14317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.639553+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:15.639553+0000","last_peered":"2024-09-11T01:21:15.639553+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:15.639553+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150040,"num_objects":5897,"num_object_clones":0,"num_object_copies":11794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12210,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14378","reported_seq":14424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.594277+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:15.594277+0000","last_peered":"2024-09-11T01:21:15.594277+0000","last_clean":"2024-09-11T01:21:15.594277+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:15.594277+0000","last_undegraded":"2024-09-11T01:21:15.594277+0000","last_fullsized":"2024-09-11T01:21:15.594277+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14378,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14430","reported_seq":14552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.431036+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:17.431036+0000","last_peered":"2024-09-11T01:21:17.431036+0000","last_clean":"2024-09-11T01:21:17.431036+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:17.431036+0000","last_undegraded":"2024-09-11T01:21:17.431036+0000","last_fullsized":"2024-09-11T01:21:17.431036+0000","mapping_epoch":31,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23949795,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11228,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14394","reported_seq":14661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.484491+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:17.484491+0000","last_peered":"2024-09-11T01:21:17.484491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:17.484491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810531,"num_objects":5814,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10882,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14858","reported_seq":15146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.543257+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:17.543257+0000","last_peered":"2024-09-11T01:21:17.543257+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:17.543257+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24388550,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11611,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1750},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14503","reported_seq":14623,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.476575+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:17.476575+0000","last_peered":"2024-09-11T01:21:17.476575+0000","last_clean":"2024-09-11T01:21:17.476575+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:17.476575+0000","last_undegraded":"2024-09-11T01:21:17.476575+0000","last_fullsized":"2024-09-11T01:21:17.476575+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11456,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14513","reported_seq":22856,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.254677+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:20.254677+0000","last_peered":"2024-09-11T01:21:20.254677+0000","last_clean":"2024-09-11T01:21:20.254677+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:20.254677+0000","last_undegraded":"2024-09-11T01:21:20.254677+0000","last_fullsized":"2024-09-11T01:21:20.254677+0000","mapping_epoch":40,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23495139,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14588","reported_seq":14866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.550155+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:17.550155+0000","last_peered":"2024-09-11T01:21:17.550155+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:17.550155+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10952,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1976}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14228","reported_seq":16044,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.707509+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:15.707509+0000","last_peered":"2024-09-11T01:21:15.707509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:15.707509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24211939,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11997,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14588","reported_seq":14742,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.724270+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:15.724270+0000","last_peered":"2024-09-11T01:21:15.724270+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:15.724270+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24469504,"num_objects":5974,"num_object_clones":0,"num_object_copies":11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12225,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14599","reported_seq":23070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.320979+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:20.320979+0000","last_peered":"2024-09-11T01:21:20.320979+0000","last_clean":"2024-09-11T01:21:20.320979+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:20.320979+0000","last_undegraded":"2024-09-11T01:21:20.320979+0000","last_fullsized":"2024-09-11T01:21:20.320979+0000","mapping_epoch":42,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23564288,"num_objects":5753,"num_object_clones":0,"num_object_copies":11506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14695","reported_seq":14980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.425268+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:17.425268+0000","last_peered":"2024-09-11T01:21:17.425268+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:17.425268+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24375296,"num_objects":5951,"num_object_clones":0,"num_object_copies":11902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11088,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14617","reported_seq":14916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.482474+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:17.482474+0000","last_peered":"2024-09-11T01:21:17.482474+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:17.482474+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11152,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14391","reported_seq":14630,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.530987+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:17.530987+0000","last_peered":"2024-09-11T01:21:17.530987+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:17.530987+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23540678,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11048,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1647},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386474742,"num_objects":94364,"num_object_clones":0,"num_object_copies":188728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94364,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":190877,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9726,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":973107200,"data_stored":972954536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622424,"kb_used_data":324168,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93749416,"statfs":{"total":96636764160,"available":95999401984,"internally_reserved":0,"allocated":331948032,"data_stored":331666204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790025,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626824,"kb_used_data":279988,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93745016,"statfs":{"total":96636764160,"available":95994896384,"internally_reserved":0,"allocated":286707712,"data_stored":286385677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":742432,"kb_used_data":349832,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93629408,"statfs":{"total":96636764160,"available":95876513792,"internally_reserved":0,"allocated":358227968,"data_stored":357946135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,5,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357003264,"data_stored":356967134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285085696,"data_stored":285009364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331018240,"data_stored":330978038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:20.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:21.351 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:21.352 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:21.377 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T01:21:20.571112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386474761,"num_objects":94366,"num_object_clones":0,"num_object_copies":188732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94366,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":190879,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9727,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1991680,"kb_used_data":953988,"kb_used_omap":12,"kb_used_meta":1037619,"kb_avail":281123840,"statfs":{"total":289910292480,"available":287870812160,"internally_reserved":0,"allocated":976883712,"data_stored":975998016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1062521863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,6,8,7,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19279872,"num_objects":-4707,"num_object_clones":0,"num_object_copies":-9414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-127,"num_objects_misplaced":-156,"num_objects_unfound":0,"num_objects_dirty":-4707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4707,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":638976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006489"},"pg_stats":[{"pgid":"2.f","version":"43'14396","reported_seq":14617,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.469121+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:17.469121+0000","last_peered":"2024-09-11T01:21:17.469121+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:17.469121+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14396,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1794}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14487","reported_seq":14568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.660442+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:15.660442+0000","last_peered":"2024-09-11T01:21:15.660442+0000","last_clean":"2024-09-11T01:21:15.660442+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:15.660442+0000","last_undegraded":"2024-09-11T01:21:15.660442+0000","last_fullsized":"2024-09-11T01:21:15.660442+0000","mapping_epoch":31,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24720326,"num_objects":6037,"num_object_clones":0,"num_object_copies":12074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12585,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14149","reported_seq":14317,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.639553+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:15.639553+0000","last_peered":"2024-09-11T01:21:15.639553+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:15.639553+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150040,"num_objects":5897,"num_object_clones":0,"num_object_copies":11794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12210,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1934}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14378","reported_seq":14424,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.594277+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:15.594277+0000","last_peered":"2024-09-11T01:21:15.594277+0000","last_clean":"2024-09-11T01:21:15.594277+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:15.594277+0000","last_undegraded":"2024-09-11T01:21:15.594277+0000","last_fullsized":"2024-09-11T01:21:15.594277+0000","mapping_epoch":31,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14378,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14430","reported_seq":14552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.431036+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:17.431036+0000","last_peered":"2024-09-11T01:21:17.431036+0000","last_clean":"2024-09-11T01:21:17.431036+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:17.431036+0000","last_undegraded":"2024-09-11T01:21:17.431036+0000","last_fullsized":"2024-09-11T01:21:17.431036+0000","mapping_epoch":31,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23949795,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11228,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14394","reported_seq":14661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.484491+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:17.484491+0000","last_peered":"2024-09-11T01:21:17.484491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:17.484491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810531,"num_objects":5814,"num_object_clones":0,"num_object_copies":11628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10882,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'14858","reported_seq":15146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.543257+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:17.543257+0000","last_peered":"2024-09-11T01:21:17.543257+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:17.543257+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24388550,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11611,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1750},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14503","reported_seq":14623,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.476575+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:17.476575+0000","last_peered":"2024-09-11T01:21:17.476575+0000","last_clean":"2024-09-11T01:21:17.476575+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:17.476575+0000","last_undegraded":"2024-09-11T01:21:17.476575+0000","last_fullsized":"2024-09-11T01:21:17.476575+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24252416,"num_objects":5921,"num_object_clones":0,"num_object_copies":11842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11456,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14513","reported_seq":22856,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.254677+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:20.254677+0000","last_peered":"2024-09-11T01:21:20.254677+0000","last_clean":"2024-09-11T01:21:20.254677+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:20.254677+0000","last_undegraded":"2024-09-11T01:21:20.254677+0000","last_fullsized":"2024-09-11T01:21:20.254677+0000","mapping_epoch":40,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23495139,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14588","reported_seq":14866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.550155+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:17.550155+0000","last_peered":"2024-09-11T01:21:17.550155+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:17.550155+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24600576,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10952,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1976}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14228","reported_seq":16044,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.707509+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:15.707509+0000","last_peered":"2024-09-11T01:21:15.707509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:15.707509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24211939,"num_objects":5912,"num_object_clones":0,"num_object_copies":11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3168,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11997,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":860,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14588","reported_seq":14742,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.724270+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:15.724270+0000","last_peered":"2024-09-11T01:21:15.724270+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:15.724270+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24469504,"num_objects":5974,"num_object_clones":0,"num_object_copies":11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12225,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1827}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14599","reported_seq":23070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.320979+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:20.320979+0000","last_peered":"2024-09-11T01:21:20.320979+0000","last_clean":"2024-09-11T01:21:20.320979+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:20.320979+0000","last_undegraded":"2024-09-11T01:21:20.320979+0000","last_fullsized":"2024-09-11T01:21:20.320979+0000","mapping_epoch":42,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23564288,"num_objects":5753,"num_object_clones":0,"num_object_copies":11506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14695","reported_seq":14980,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.425268+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:17.425268+0000","last_peered":"2024-09-11T01:21:17.425268+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:17.425268+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24375296,"num_objects":5951,"num_object_clones":0,"num_object_copies":11902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11088,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14617","reported_seq":14916,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.482474+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:17.482474+0000","last_peered":"2024-09-11T01:21:17.482474+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:17.482474+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24506368,"num_objects":5983,"num_object_clones":0,"num_object_copies":11966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11152,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14391","reported_seq":14630,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.530987+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:17.530987+0000","last_peered":"2024-09-11T01:21:17.530987+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:17.530987+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23540678,"num_objects":5749,"num_object_clones":0,"num_object_copies":11498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5749,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11048,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1647},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386474742,"num_objects":94364,"num_object_clones":0,"num_object_copies":188728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94364,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":190877,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9726,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":973107200,"data_stored":972954536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622424,"kb_used_data":324168,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93749416,"statfs":{"total":96636764160,"available":95999401984,"internally_reserved":0,"allocated":331948032,"data_stored":331666204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790025,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626824,"kb_used_data":279988,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93745016,"statfs":{"total":96636764160,"available":95994896384,"internally_reserved":0,"allocated":286707712,"data_stored":286385677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":742432,"kb_used_data":349832,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93629408,"statfs":{"total":96636764160,"available":95876513792,"internally_reserved":0,"allocated":358227968,"data_stored":357946135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,1,1,5,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357003264,"data_stored":356967134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285085696,"data_stored":285009364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331018240,"data_stored":330978038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14396", 'reported_seq': 14617, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.469121+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:17.469121+0000', 'last_peered': '2024-09-11T01:21:17.469121+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:17.469121+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23928832, 'num_objects': 5842, 'num_object_clones': 0, 'num_object_copies': 11684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5842, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14396, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1794}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'14149", 'reported_seq': 14317, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:15.639553+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:15.639553+0000', 'last_peered': '2024-09-11T01:21:15.639553+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:15.639553+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24150040, 'num_objects': 5897, 'num_object_clones': 0, 'num_object_copies': 11794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5897, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12210, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1934}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14394", 'reported_seq': 14661, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.484491+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:17.484491+0000', 'last_peered': '2024-09-11T01:21:17.484491+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:17.484491+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23810531, 'num_objects': 5814, 'num_object_clones': 0, 'num_object_copies': 11628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5814, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10882, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1765}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'14858", 'reported_seq': 15146, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.543257+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:17.543257+0000', 'last_peered': '2024-09-11T01:21:17.543257+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:17.543257+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24388550, 'num_objects': 5956, 'num_object_clones': 0, 'num_object_copies': 11912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5956, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11611, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1750}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14588", 'reported_seq': 14866, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.550155+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:17.550155+0000', 'last_peered': '2024-09-11T01:21:17.550155+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:17.550155+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24600576, 'num_objects': 6006, 'num_object_clones': 0, 'num_object_copies': 12012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10952, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1976}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'14228", 'reported_seq': 16044, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:15.707509+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:15.707509+0000', 'last_peered': '2024-09-11T01:21:15.707509+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:15.707509+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'4200", 'ondisk_log_start': "39'4200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24211939, 'num_objects': 5912, 'num_object_clones': 0, 'num_object_copies': 11824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3168, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5912, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11997, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 860, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2744}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'14588", 'reported_seq': 14742, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:15.724270+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:15.724270+0000', 'last_peered': '2024-09-11T01:21:15.724270+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:15.724270+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24469504, 'num_objects': 5974, 'num_object_clones': 0, 'num_object_copies': 11948, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5974, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12225, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1827}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'14695", 'reported_seq': 14980, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.425268+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:17.425268+0000', 'last_peered': '2024-09-11T01:21:17.425268+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:17.425268+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'4600", 'ondisk_log_start': "39'4600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24375296, 'num_objects': 5951, 'num_object_clones': 0, 'num_object_copies': 11902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11088, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14617", 'reported_seq': 14916, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.482474+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:17.482474+0000', 'last_peered': '2024-09-11T01:21:17.482474+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:17.482474+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'4600", 'ondisk_log_start': "39'4600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24506368, 'num_objects': 5983, 'num_object_clones': 0, 'num_object_copies': 11966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11152, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:21.380 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14391", 'reported_seq': 14630, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:17.530987+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:17.530987+0000', 'last_peered': '2024-09-11T01:21:17.530987+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:17.530987+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'4300", 'ondisk_log_start': "39'4300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23540678, 'num_objects': 5749, 'num_object_clones': 0, 'num_object_copies': 11498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5749, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11048, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1647}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:21.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:22.121 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:22.121 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39908/188732 objects degraded (21.145%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":482,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94366,"data_bytes":386474761,"bytes_used":2039480320,"bytes_avail":287870812160,"bytes_total":289910292480,"degraded_objects":39908,"degraded_total":188732,"degraded_ratio":0.21145327766356525,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":53219,"recovering_keys_per_sec":0,"num_objects_recovered":283,"num_bytes_recovered":638976,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":392},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:22.149 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:22.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:22.777 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:22.778 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:22.802 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-11T01:21:22.571787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":378168073,"num_objects":92338,"num_object_clones":0,"num_object_copies":184676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92338,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":192907,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9810,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1988312,"kb_used_data":942304,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281127208,"statfs":{"total":289910292480,"available":287874260992,"internally_reserved":0,"allocated":964919296,"data_stored":964033600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1071041543},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,6,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26542080,"num_objects":-6480,"num_object_clones":0,"num_object_copies":-12960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":-156,"num_objects_unfound":0,"num_objects_dirty":-6480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6480,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":638976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006680"},"pg_stats":[{"pgid":"2.f","version":"43'14538","reported_seq":14761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.509336+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:22.509336+0000","last_peered":"2024-09-11T01:21:22.509336+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:22.509336+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14648","reported_seq":14729,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.661622+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:20.661622+0000","last_peered":"2024-09-11T01:21:20.661622+0000","last_clean":"2024-09-11T01:21:20.661622+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:20.661622+0000","last_undegraded":"2024-09-11T01:21:20.661622+0000","last_fullsized":"2024-09-11T01:21:20.661622+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24060870,"num_objects":5876,"num_object_clones":0,"num_object_copies":11752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5876,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12746,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14287","reported_seq":14455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.687717+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:20.687717+0000","last_peered":"2024-09-11T01:21:20.687717+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:20.687717+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23584792,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12348,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1796}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14516","reported_seq":14562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.645070+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:20.645070+0000","last_peered":"2024-09-11T01:21:20.645070+0000","last_clean":"2024-09-11T01:21:20.645070+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:20.645070+0000","last_undegraded":"2024-09-11T01:21:20.645070+0000","last_fullsized":"2024-09-11T01:21:20.645070+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14516,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14583","reported_seq":14705,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.526064+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:22.526064+0000","last_peered":"2024-09-11T01:21:22.526064+0000","last_clean":"2024-09-11T01:21:22.526064+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:22.526064+0000","last_undegraded":"2024-09-11T01:21:22.526064+0000","last_fullsized":"2024-09-11T01:21:22.526064+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23323107,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11381,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14532","reported_seq":14795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.512431+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:22.512431+0000","last_peered":"2024-09-11T01:21:22.512431+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:22.512431+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23245283,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11020,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15019","reported_seq":15307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.481891+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:22.481891+0000","last_peered":"2024-09-11T01:21:22.481891+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:22.481891+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23729094,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11772,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1589},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14629","reported_seq":14749,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.473811+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:22.473811+0000","last_peered":"2024-09-11T01:21:22.473811+0000","last_clean":"2024-09-11T01:21:22.473811+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:22.473811+0000","last_undegraded":"2024-09-11T01:21:22.473811+0000","last_fullsized":"2024-09-11T01:21:22.473811+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14513","reported_seq":22856,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.254677+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:20.254677+0000","last_peered":"2024-09-11T01:21:20.254677+0000","last_clean":"2024-09-11T01:21:20.254677+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:20.254677+0000","last_undegraded":"2024-09-11T01:21:20.254677+0000","last_fullsized":"2024-09-11T01:21:20.254677+0000","mapping_epoch":40,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23495139,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14731","reported_seq":15009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.513526+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:22.513526+0000","last_peered":"2024-09-11T01:21:22.513526+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:22.513526+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14366","reported_seq":16348,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.733739+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:20.733739+0000","last_peered":"2024-09-11T01:21:20.733739+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:20.733739+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23646691,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12135,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2689}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14728","reported_seq":14882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.672990+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:20.672990+0000","last_peered":"2024-09-11T01:21:20.672990+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:20.672990+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12365,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1687}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14599","reported_seq":23070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.320979+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:20.320979+0000","last_peered":"2024-09-11T01:21:20.320979+0000","last_clean":"2024-09-11T01:21:20.320979+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:20.320979+0000","last_undegraded":"2024-09-11T01:21:20.320979+0000","last_fullsized":"2024-09-11T01:21:20.320979+0000","mapping_epoch":42,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23564288,"num_objects":5753,"num_object_clones":0,"num_object_copies":11506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14849","reported_seq":15134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.441674+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:22.441674+0000","last_peered":"2024-09-11T01:21:22.441674+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:22.441674+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11242,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14773","reported_seq":15073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.497823+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:22.497823+0000","last_peered":"2024-09-11T01:21:22.497823+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:22.497823+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14531","reported_seq":14770,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.519911+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:22.519911+0000","last_peered":"2024-09-11T01:21:22.519911+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:22.519911+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22967238,"num_objects":5609,"num_object_clones":0,"num_object_copies":11218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11188,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1507},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":378168054,"num_objects":92336,"num_object_clones":0,"num_object_copies":184672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92336,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":192905,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9809,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":961339392,"data_stored":961186728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621984,"kb_used_data":319564,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93749856,"statfs":{"total":96636764160,"available":95999852544,"internally_reserved":0,"allocated":327233536,"data_stored":326951708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,0,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790025,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626824,"kb_used_data":279988,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93745016,"statfs":{"total":96636764160,"available":95994896384,"internally_reserved":0,"allocated":286707712,"data_stored":286385677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739504,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93632336,"statfs":{"total":96636764160,"available":95879512064,"internally_reserved":0,"allocated":350978048,"data_stored":350696215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349843456,"data_stored":349807326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285085696,"data_stored":285009364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326410240,"data_stored":326370038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:25.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:26.379 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:26.380 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:26.415 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-11T01:21:24.572684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":378168073,"num_objects":92338,"num_object_clones":0,"num_object_copies":184676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92338,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":192907,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9810,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1988312,"kb_used_data":942304,"kb_used_omap":12,"kb_used_meta":1045939,"kb_avail":281127208,"statfs":{"total":289910292480,"available":287874260992,"internally_reserved":0,"allocated":964919296,"data_stored":964033600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1071041543},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,5,6,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18890752,"num_objects":-4612,"num_object_clones":0,"num_object_copies":-9224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":-4612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4612,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006945"},"pg_stats":[{"pgid":"2.f","version":"43'14538","reported_seq":14761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.509336+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:22.509336+0000","last_peered":"2024-09-11T01:21:22.509336+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:22.509336+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14648","reported_seq":14729,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.661622+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:20.661622+0000","last_peered":"2024-09-11T01:21:20.661622+0000","last_clean":"2024-09-11T01:21:20.661622+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:20.661622+0000","last_undegraded":"2024-09-11T01:21:20.661622+0000","last_fullsized":"2024-09-11T01:21:20.661622+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24060870,"num_objects":5876,"num_object_clones":0,"num_object_copies":11752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5876,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12746,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14287","reported_seq":14455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.687717+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:20.687717+0000","last_peered":"2024-09-11T01:21:20.687717+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:20.687717+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4200","ondisk_log_start":"39'4200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23584792,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12348,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1796}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14516","reported_seq":14562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.645070+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:20.645070+0000","last_peered":"2024-09-11T01:21:20.645070+0000","last_clean":"2024-09-11T01:21:20.645070+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:20.645070+0000","last_undegraded":"2024-09-11T01:21:20.645070+0000","last_fullsized":"2024-09-11T01:21:20.645070+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23945216,"num_objects":5846,"num_object_clones":0,"num_object_copies":11692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14516,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14583","reported_seq":14705,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.526064+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:22.526064+0000","last_peered":"2024-09-11T01:21:22.526064+0000","last_clean":"2024-09-11T01:21:22.526064+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:22.526064+0000","last_undegraded":"2024-09-11T01:21:22.526064+0000","last_fullsized":"2024-09-11T01:21:22.526064+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23323107,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11381,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14532","reported_seq":14795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.512431+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:22.512431+0000","last_peered":"2024-09-11T01:21:22.512431+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:22.512431+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23245283,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11020,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15019","reported_seq":15307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.481891+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:22.481891+0000","last_peered":"2024-09-11T01:21:22.481891+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:22.481891+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23729094,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11772,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1589},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14629","reported_seq":14749,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.473811+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:22.473811+0000","last_peered":"2024-09-11T01:21:22.473811+0000","last_clean":"2024-09-11T01:21:22.473811+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:22.473811+0000","last_undegraded":"2024-09-11T01:21:22.473811+0000","last_fullsized":"2024-09-11T01:21:22.473811+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14513","reported_seq":22856,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.254677+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:20.254677+0000","last_peered":"2024-09-11T01:21:20.254677+0000","last_clean":"2024-09-11T01:21:20.254677+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:20.254677+0000","last_undegraded":"2024-09-11T01:21:20.254677+0000","last_fullsized":"2024-09-11T01:21:20.254677+0000","mapping_epoch":40,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23495139,"num_objects":5737,"num_object_clones":0,"num_object_copies":11474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14731","reported_seq":15009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.513526+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:22.513526+0000","last_peered":"2024-09-11T01:21:22.513526+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:22.513526+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14366","reported_seq":16348,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.733739+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:20.733739+0000","last_peered":"2024-09-11T01:21:20.733739+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:20.733739+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4300","ondisk_log_start":"39'4300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23646691,"num_objects":5774,"num_object_clones":0,"num_object_copies":11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3085,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12135,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2689}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14728","reported_seq":14882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.672990+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:20.672990+0000","last_peered":"2024-09-11T01:21:20.672990+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:20.672990+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23896064,"num_objects":5834,"num_object_clones":0,"num_object_copies":11668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12365,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1687}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14599","reported_seq":23070,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.320979+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:20.320979+0000","last_peered":"2024-09-11T01:21:20.320979+0000","last_clean":"2024-09-11T01:21:20.320979+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:20.320979+0000","last_undegraded":"2024-09-11T01:21:20.320979+0000","last_fullsized":"2024-09-11T01:21:20.320979+0000","mapping_epoch":42,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23564288,"num_objects":5753,"num_object_clones":0,"num_object_copies":11506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11289,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14849","reported_seq":15134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.441674+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:22.441674+0000","last_peered":"2024-09-11T01:21:22.441674+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:22.441674+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11242,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14773","reported_seq":15073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.497823+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:22.497823+0000","last_peered":"2024-09-11T01:21:22.497823+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:22.497823+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14531","reported_seq":14770,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.519911+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:22.519911+0000","last_peered":"2024-09-11T01:21:22.519911+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:22.519911+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22967238,"num_objects":5609,"num_object_clones":0,"num_object_copies":11218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11188,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1507},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":378168054,"num_objects":92336,"num_object_clones":0,"num_object_copies":184672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92336,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":192905,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9809,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":961339392,"data_stored":961186728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621984,"kb_used_data":319564,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93749856,"statfs":{"total":96636764160,"available":95999852544,"internally_reserved":0,"allocated":327233536,"data_stored":326951708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,0,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790025,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626824,"kb_used_data":279988,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93745016,"statfs":{"total":96636764160,"available":95994896384,"internally_reserved":0,"allocated":286707712,"data_stored":286385677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,3,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739504,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93632336,"statfs":{"total":96636764160,"available":95879512064,"internally_reserved":0,"allocated":350978048,"data_stored":350696215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349843456,"data_stored":349807326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285085696,"data_stored":285009364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326410240,"data_stored":326370038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:26.417 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:27.008 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:27.011 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:27.035 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-11T01:21:26.573611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374145801,"num_objects":91356,"num_object_clones":0,"num_object_copies":182712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91356,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":193889,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9893,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1990744,"kb_used_data":930848,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281124776,"statfs":{"total":289910292480,"available":287871770624,"internally_reserved":0,"allocated":953188352,"data_stored":952302656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,8,12,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22913024,"num_objects":-5594,"num_object_clones":0,"num_object_copies":-11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":-5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007110"},"pg_stats":[{"pgid":"2.f","version":"43'14538","reported_seq":14761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.509336+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:22.509336+0000","last_peered":"2024-09-11T01:21:22.509336+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:22.509336+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14785","reported_seq":14866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.647758+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:25.647758+0000","last_peered":"2024-09-11T01:21:25.647758+0000","last_clean":"2024-09-11T01:21:25.647758+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:25.647758+0000","last_undegraded":"2024-09-11T01:21:25.647758+0000","last_fullsized":"2024-09-11T01:21:25.647758+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23499718,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12883,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14431","reported_seq":14599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.738367+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:25.738367+0000","last_peered":"2024-09-11T01:21:25.738367+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:25.738367+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994968,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12492,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14656","reported_seq":14702,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.694680+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:25.694680+0000","last_peered":"2024-09-11T01:21:25.694680+0000","last_clean":"2024-09-11T01:21:25.694680+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:25.694680+0000","last_undegraded":"2024-09-11T01:21:25.694680+0000","last_fullsized":"2024-09-11T01:21:25.694680+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14656,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14583","reported_seq":14705,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.526064+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:22.526064+0000","last_peered":"2024-09-11T01:21:22.526064+0000","last_clean":"2024-09-11T01:21:22.526064+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:22.526064+0000","last_undegraded":"2024-09-11T01:21:22.526064+0000","last_fullsized":"2024-09-11T01:21:22.526064+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23323107,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11381,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14532","reported_seq":14795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.512431+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:22.512431+0000","last_peered":"2024-09-11T01:21:22.512431+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:22.512431+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23245283,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11020,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15019","reported_seq":15307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.481891+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:22.481891+0000","last_peered":"2024-09-11T01:21:22.481891+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:22.481891+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23729094,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11772,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1589},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14629","reported_seq":14749,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.473811+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:22.473811+0000","last_peered":"2024-09-11T01:21:22.473811+0000","last_clean":"2024-09-11T01:21:22.473811+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:22.473811+0000","last_undegraded":"2024-09-11T01:21:22.473811+0000","last_fullsized":"2024-09-11T01:21:22.473811+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14650","reported_seq":22993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.338131+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:25.338131+0000","last_peered":"2024-09-11T01:21:25.338131+0000","last_clean":"2024-09-11T01:21:25.338131+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:25.338131+0000","last_undegraded":"2024-09-11T01:21:25.338131+0000","last_fullsized":"2024-09-11T01:21:25.338131+0000","mapping_epoch":40,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933987,"num_objects":5600,"num_object_clones":0,"num_object_copies":11200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12517,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14731","reported_seq":15009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.513526+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:22.513526+0000","last_peered":"2024-09-11T01:21:22.513526+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:22.513526+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14512","reported_seq":16660,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.747717+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:25.747717+0000","last_peered":"2024-09-11T01:21:25.747717+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:25.747717+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23048675,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12281,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14890","reported_seq":15044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.683858+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:25.683858+0000","last_peered":"2024-09-11T01:21:25.683858+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:25.683858+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1525}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14715","reported_seq":23185,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.351991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:25.351991+0000","last_peered":"2024-09-11T01:21:25.351991+0000","last_clean":"2024-09-11T01:21:25.351991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:25.351991+0000","last_undegraded":"2024-09-11T01:21:25.351991+0000","last_fullsized":"2024-09-11T01:21:25.351991+0000","mapping_epoch":42,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11405,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14849","reported_seq":15134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.441674+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:22.441674+0000","last_peered":"2024-09-11T01:21:22.441674+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:22.441674+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11242,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14773","reported_seq":15073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.497823+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:22.497823+0000","last_peered":"2024-09-11T01:21:22.497823+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:22.497823+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14531","reported_seq":14770,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.519911+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:22.519911+0000","last_peered":"2024-09-11T01:21:22.519911+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:22.519911+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22967238,"num_objects":5609,"num_object_clones":0,"num_object_copies":11218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11188,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1507},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374145782,"num_objects":91354,"num_object_clones":0,"num_object_copies":182708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91354,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":193887,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9892,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949497856,"data_stored":949345192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621596,"kb_used_data":315016,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93750244,"statfs":{"total":96636764160,"available":96000249856,"internally_reserved":0,"allocated":322576384,"data_stored":322294556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790026,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629644,"kb_used_data":273080,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93742196,"statfs":{"total":96636764160,"available":95992008704,"internally_reserved":0,"allocated":279633920,"data_stored":279311885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739504,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93632336,"statfs":{"total":96636764160,"available":95879512064,"internally_reserved":0,"allocated":350978048,"data_stored":350696215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349843456,"data_stored":349807326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278016000,"data_stored":277939668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321638400,"data_stored":321598198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14538", 'reported_seq': 14761, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.509336+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:22.509336+0000', 'last_peered': '2024-09-11T01:21:22.509336+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:22.509336+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23347200, 'num_objects': 5700, 'num_object_clones': 0, 'num_object_copies': 11400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14538, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1652}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'14431", 'reported_seq': 14599, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:25.738367+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:25.738367+0000', 'last_peered': '2024-09-11T01:21:25.738367+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:25.738367+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'4400", 'ondisk_log_start': "39'4400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22994968, 'num_objects': 5615, 'num_object_clones': 0, 'num_object_copies': 11230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5615, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12492, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1652}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14532", 'reported_seq': 14795, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.512431+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:22.512431+0000', 'last_peered': '2024-09-11T01:21:22.512431+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:22.512431+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23245283, 'num_objects': 5676, 'num_object_clones': 0, 'num_object_copies': 11352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5676, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11020, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1627}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'15019", 'reported_seq': 15307, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.481891+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:22.481891+0000', 'last_peered': '2024-09-11T01:21:22.481891+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:22.481891+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23729094, 'num_objects': 5795, 'num_object_clones': 0, 'num_object_copies': 11590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5795, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11772, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1589}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.037 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14731", 'reported_seq': 15009, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.513526+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:22.513526+0000', 'last_peered': '2024-09-11T01:21:22.513526+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:22.513526+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'4700", 'ondisk_log_start': "39'4700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24014848, 'num_objects': 5863, 'num_object_clones': 0, 'num_object_copies': 11726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11095, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1833}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'14512", 'reported_seq': 16660, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:25.747717+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:25.747717+0000', 'last_peered': '2024-09-11T01:21:25.747717+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:25.747717+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23048675, 'num_objects': 5628, 'num_object_clones': 0, 'num_object_copies': 11256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5628, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12281, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1026, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2626}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'14890", 'reported_seq': 15044, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:25.683858+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:25.683858+0000', 'last_peered': '2024-09-11T01:21:25.683858+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:25.683858+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23232512, 'num_objects': 5672, 'num_object_clones': 0, 'num_object_copies': 11344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12527, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1525}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'14849", 'reported_seq': 15134, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.441674+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:22.441674+0000', 'last_peered': '2024-09-11T01:21:22.441674+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:22.441674+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23744512, 'num_objects': 5797, 'num_object_clones': 0, 'num_object_copies': 11594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11242, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1680}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14773", 'reported_seq': 15073, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.497823+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:22.497823+0000', 'last_peered': '2024-09-11T01:21:22.497823+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:22.497823+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'4700", 'ondisk_log_start': "39'4700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23867392, 'num_objects': 5827, 'num_object_clones': 0, 'num_object_copies': 11654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11308, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1762}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:27.038 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14531", 'reported_seq': 14770, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.519911+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:22.519911+0000', 'last_peered': '2024-09-11T01:21:22.519911+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:22.519911+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22967238, 'num_objects': 5609, 'num_object_clones': 0, 'num_object_copies': 11218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5609, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11188, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1507}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:27.039 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:27.763 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:27.763 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39742/182712 objects degraded (21.751%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":488,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91356,"data_bytes":374145801,"bytes_used":2038521856,"bytes_avail":287871770624,"bytes_total":289910292480,"degraded_objects":39742,"degraded_total":182712,"degraded_ratio":0.21751171242173478,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":3411,"recovering_keys_per_sec":0,"num_objects_recovered":279,"num_bytes_recovered":40960,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:27.784 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:27.785 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:28.363 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:28.365 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:28.387 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-11T01:21:26.573611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374145801,"num_objects":91356,"num_object_clones":0,"num_object_copies":182712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91356,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":193889,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9893,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1990744,"kb_used_data":930848,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281124776,"statfs":{"total":289910292480,"available":287871770624,"internally_reserved":0,"allocated":953188352,"data_stored":952302656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,2,8,12,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22913024,"num_objects":-5594,"num_object_clones":0,"num_object_copies":-11188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":-10,"num_objects_unfound":0,"num_objects_dirty":-5594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":40960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007110"},"pg_stats":[{"pgid":"2.f","version":"43'14538","reported_seq":14761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.509336+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:22.509336+0000","last_peered":"2024-09-11T01:21:22.509336+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:22.509336+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1652}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14785","reported_seq":14866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.647758+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:25.647758+0000","last_peered":"2024-09-11T01:21:25.647758+0000","last_clean":"2024-09-11T01:21:25.647758+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:25.647758+0000","last_undegraded":"2024-09-11T01:21:25.647758+0000","last_fullsized":"2024-09-11T01:21:25.647758+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23499718,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12883,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14431","reported_seq":14599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.738367+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:25.738367+0000","last_peered":"2024-09-11T01:21:25.738367+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:25.738367+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994968,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12492,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14656","reported_seq":14702,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.694680+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:25.694680+0000","last_peered":"2024-09-11T01:21:25.694680+0000","last_clean":"2024-09-11T01:21:25.694680+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:25.694680+0000","last_undegraded":"2024-09-11T01:21:25.694680+0000","last_fullsized":"2024-09-11T01:21:25.694680+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14656,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14583","reported_seq":14705,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.526064+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:22.526064+0000","last_peered":"2024-09-11T01:21:22.526064+0000","last_clean":"2024-09-11T01:21:22.526064+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:22.526064+0000","last_undegraded":"2024-09-11T01:21:22.526064+0000","last_fullsized":"2024-09-11T01:21:22.526064+0000","mapping_epoch":31,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23323107,"num_objects":5695,"num_object_clones":0,"num_object_copies":11390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11381,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14532","reported_seq":14795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.512431+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:22.512431+0000","last_peered":"2024-09-11T01:21:22.512431+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:22.512431+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23245283,"num_objects":5676,"num_object_clones":0,"num_object_copies":11352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5676,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11020,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1627},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15019","reported_seq":15307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.481891+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:22.481891+0000","last_peered":"2024-09-11T01:21:22.481891+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:22.481891+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23729094,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11772,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1589},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14629","reported_seq":14749,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.473811+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:22.473811+0000","last_peered":"2024-09-11T01:21:22.473811+0000","last_clean":"2024-09-11T01:21:22.473811+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:22.473811+0000","last_undegraded":"2024-09-11T01:21:22.473811+0000","last_fullsized":"2024-09-11T01:21:22.473811+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11582,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14650","reported_seq":22993,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.338131+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:25.338131+0000","last_peered":"2024-09-11T01:21:25.338131+0000","last_clean":"2024-09-11T01:21:25.338131+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:25.338131+0000","last_undegraded":"2024-09-11T01:21:25.338131+0000","last_fullsized":"2024-09-11T01:21:25.338131+0000","mapping_epoch":40,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22933987,"num_objects":5600,"num_object_clones":0,"num_object_copies":11200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12517,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14731","reported_seq":15009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.513526+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:22.513526+0000","last_peered":"2024-09-11T01:21:22.513526+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:22.513526+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24014848,"num_objects":5863,"num_object_clones":0,"num_object_copies":11726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14512","reported_seq":16660,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.747717+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:25.747717+0000","last_peered":"2024-09-11T01:21:25.747717+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:25.747717+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23048675,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12281,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14890","reported_seq":15044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.683858+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:25.683858+0000","last_peered":"2024-09-11T01:21:25.683858+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:25.683858+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1525}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14715","reported_seq":23185,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.351991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:25.351991+0000","last_peered":"2024-09-11T01:21:25.351991+0000","last_clean":"2024-09-11T01:21:25.351991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:25.351991+0000","last_undegraded":"2024-09-11T01:21:25.351991+0000","last_fullsized":"2024-09-11T01:21:25.351991+0000","mapping_epoch":42,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23089152,"num_objects":5637,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11405,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'14849","reported_seq":15134,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.441674+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:22.441674+0000","last_peered":"2024-09-11T01:21:22.441674+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:22.441674+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23744512,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11242,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14773","reported_seq":15073,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.497823+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:22.497823+0000","last_peered":"2024-09-11T01:21:22.497823+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:22.497823+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23867392,"num_objects":5827,"num_object_clones":0,"num_object_copies":11654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11308,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14531","reported_seq":14770,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.519911+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:22.519911+0000","last_peered":"2024-09-11T01:21:22.519911+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:22.519911+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22967238,"num_objects":5609,"num_object_clones":0,"num_object_copies":11218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11188,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1507},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374145782,"num_objects":91354,"num_object_clones":0,"num_object_copies":182708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91354,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":193887,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9892,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949497856,"data_stored":949345192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621596,"kb_used_data":315016,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93750244,"statfs":{"total":96636764160,"available":96000249856,"internally_reserved":0,"allocated":322576384,"data_stored":322294556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790026,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629644,"kb_used_data":273080,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93742196,"statfs":{"total":96636764160,"available":95992008704,"internally_reserved":0,"allocated":279633920,"data_stored":279311885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739504,"kb_used_data":342752,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93632336,"statfs":{"total":96636764160,"available":95879512064,"internally_reserved":0,"allocated":350978048,"data_stored":350696215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":349843456,"data_stored":349807326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":278016000,"data_stored":277939668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321638400,"data_stored":321598198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:31.389 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:31.992 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:31.993 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:32.023 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-11T01:21:30.575463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":367571721,"num_objects":89751,"num_object_clones":0,"num_object_copies":179502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89751,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":195494,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9893,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1985308,"kb_used_data":917068,"kb_used_omap":12,"kb_used_meta":1068147,"kb_avail":281130212,"statfs":{"total":289910292480,"available":287877337088,"internally_reserved":0,"allocated":939077632,"data_stored":938191936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1093782535},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,0,6,7,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20103168,"num_objects":-4908,"num_object_clones":0,"num_object_copies":-9816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4908,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005134"},"pg_stats":[{"pgid":"2.f","version":"43'14680","reported_seq":14904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.563128+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:27.563128+0000","last_peered":"2024-09-11T01:21:27.563128+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:27.563128+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22765568,"num_objects":5558,"num_object_clones":0,"num_object_copies":11116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14785","reported_seq":14866,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.647758+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:25.647758+0000","last_peered":"2024-09-11T01:21:25.647758+0000","last_clean":"2024-09-11T01:21:25.647758+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:25.647758+0000","last_undegraded":"2024-09-11T01:21:25.647758+0000","last_fullsized":"2024-09-11T01:21:25.647758+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23499718,"num_objects":5739,"num_object_clones":0,"num_object_copies":11478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5739,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12883,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14431","reported_seq":14599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.738367+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:25.738367+0000","last_peered":"2024-09-11T01:21:25.738367+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:25.738367+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4400","ondisk_log_start":"39'4400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22994968,"num_objects":5615,"num_object_clones":0,"num_object_copies":11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12492,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1652}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14656","reported_seq":14702,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.694680+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:25.694680+0000","last_peered":"2024-09-11T01:21:25.694680+0000","last_clean":"2024-09-11T01:21:25.694680+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:25.694680+0000","last_undegraded":"2024-09-11T01:21:25.694680+0000","last_fullsized":"2024-09-11T01:21:25.694680+0000","mapping_epoch":31,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14656,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14719","reported_seq":14841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.492972+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:27.492972+0000","last_peered":"2024-09-11T01:21:27.492972+0000","last_clean":"2024-09-11T01:21:27.492972+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:27.492972+0000","last_undegraded":"2024-09-11T01:21:27.492972+0000","last_fullsized":"2024-09-11T01:21:27.492972+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22766051,"num_objects":5559,"num_object_clones":0,"num_object_copies":11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11517,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14655","reported_seq":14922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.476254+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:27.476254+0000","last_peered":"2024-09-11T01:21:27.476254+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:27.476254+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22741475,"num_objects":5553,"num_object_clones":0,"num_object_copies":11106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11143,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1504},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15172","reported_seq":15459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.503428+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:27.503428+0000","last_peered":"2024-09-11T01:21:27.503428+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:27.503428+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23102406,"num_objects":5642,"num_object_clones":0,"num_object_copies":11284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5642,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11925,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1436},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14795","reported_seq":14915,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.552665+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:27.552665+0000","last_peered":"2024-09-11T01:21:27.552665+0000","last_clean":"2024-09-11T01:21:27.552665+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:27.552665+0000","last_undegraded":"2024-09-11T01:21:27.552665+0000","last_fullsized":"2024-09-11T01:21:27.552665+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11748,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14776","reported_seq":23119,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.350482+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:30.350482+0000","last_peered":"2024-09-11T01:21:30.350482+0000","last_clean":"2024-09-11T01:21:30.350482+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:30.350482+0000","last_undegraded":"2024-09-11T01:21:30.350482+0000","last_fullsized":"2024-09-11T01:21:30.350482+0000","mapping_epoch":40,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22417891,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12643,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14897","reported_seq":15175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.570540+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:27.570540+0000","last_peered":"2024-09-11T01:21:27.570540+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:27.570540+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11261,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14512","reported_seq":16660,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.747717+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:25.747717+0000","last_peered":"2024-09-11T01:21:25.747717+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:25.747717+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23048675,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12281,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2626}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'14890","reported_seq":15044,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.683858+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:25.683858+0000","last_peered":"2024-09-11T01:21:25.683858+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:25.683858+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23232512,"num_objects":5672,"num_object_clones":0,"num_object_copies":11344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12527,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1525}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14864","reported_seq":23335,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.281781+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:30.281781+0000","last_peered":"2024-09-11T01:21:30.281781+0000","last_clean":"2024-09-11T01:21:30.281781+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:30.281781+0000","last_undegraded":"2024-09-11T01:21:30.281781+0000","last_fullsized":"2024-09-11T01:21:30.281781+0000","mapping_epoch":42,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15006","reported_seq":15291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.527476+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:27.527476+0000","last_peered":"2024-09-11T01:21:27.527476+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:27.527476+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23101440,"num_objects":5640,"num_object_clones":0,"num_object_copies":11280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11399,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14935","reported_seq":15235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.453907+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:27.453907+0000","last_peered":"2024-09-11T01:21:27.453907+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:27.453907+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11470,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14656","reported_seq":14896,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.576744+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:27.576744+0000","last_peered":"2024-09-11T01:21:27.576744+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:27.576744+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22455238,"num_objects":5484,"num_object_clones":0,"num_object_copies":10968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11313,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":367571702,"num_objects":89749,"num_object_clones":0,"num_object_copies":179498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89749,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":195492,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9892,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":935325696,"data_stored":935173032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621596,"kb_used_data":315016,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93750244,"statfs":{"total":96636764160,"available":96000249856,"internally_reserved":0,"allocated":322576384,"data_stored":322294556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626876,"kb_used_data":266152,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93744964,"statfs":{"total":96636764160,"available":95994843136,"internally_reserved":0,"allocated":272539648,"data_stored":272217613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736836,"kb_used_data":335900,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93635004,"statfs":{"total":96636764160,"available":95882244096,"internally_reserved":0,"allocated":343961600,"data_stored":343679767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342650880,"data_stored":342614750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":271036416,"data_stored":270960084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321638400,"data_stored":321598198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:32.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:32.642 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:32.642 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:32.676 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-11T01:21:32.576382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364593929,"num_objects":89024,"num_object_clones":0,"num_object_copies":178048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89024,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":196221,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9977,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1984844,"kb_used_data":912448,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281130676,"statfs":{"total":289910292480,"available":287877812224,"internally_reserved":0,"allocated":934346752,"data_stored":933461056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1098042375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,7,6,7,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21880832,"num_objects":-5342,"num_object_clones":0,"num_object_copies":-10684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005269"},"pg_stats":[{"pgid":"2.f","version":"43'14680","reported_seq":14904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.563128+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:27.563128+0000","last_peered":"2024-09-11T01:21:27.563128+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:27.563128+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22765568,"num_objects":5558,"num_object_clones":0,"num_object_copies":11116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14924","reported_seq":15005,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.789535+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:30.789535+0000","last_peered":"2024-09-11T01:21:30.789535+0000","last_clean":"2024-09-11T01:21:30.789535+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:30.789535+0000","last_undegraded":"2024-09-11T01:21:30.789535+0000","last_fullsized":"2024-09-11T01:21:30.789535+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22930374,"num_objects":5600,"num_object_clones":0,"num_object_copies":11200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13022,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14569","reported_seq":14737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.773418+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:30.773418+0000","last_peered":"2024-09-11T01:21:30.773418+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:30.773418+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429720,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12630,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14819","reported_seq":14865,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.784374+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:30.784374+0000","last_peered":"2024-09-11T01:21:30.784374+0000","last_clean":"2024-09-11T01:21:30.784374+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:30.784374+0000","last_undegraded":"2024-09-11T01:21:30.784374+0000","last_fullsized":"2024-09-11T01:21:30.784374+0000","mapping_epoch":31,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14719","reported_seq":14841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.492972+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:27.492972+0000","last_peered":"2024-09-11T01:21:27.492972+0000","last_clean":"2024-09-11T01:21:27.492972+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:27.492972+0000","last_undegraded":"2024-09-11T01:21:27.492972+0000","last_fullsized":"2024-09-11T01:21:27.492972+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22766051,"num_objects":5559,"num_object_clones":0,"num_object_copies":11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11517,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14655","reported_seq":14922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.476254+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:27.476254+0000","last_peered":"2024-09-11T01:21:27.476254+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:27.476254+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22741475,"num_objects":5553,"num_object_clones":0,"num_object_copies":11106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11143,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1504},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15172","reported_seq":15459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.503428+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:27.503428+0000","last_peered":"2024-09-11T01:21:27.503428+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:27.503428+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23102406,"num_objects":5642,"num_object_clones":0,"num_object_copies":11284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5642,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11925,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1436},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14795","reported_seq":14915,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.552665+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:27.552665+0000","last_peered":"2024-09-11T01:21:27.552665+0000","last_clean":"2024-09-11T01:21:27.552665+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:27.552665+0000","last_undegraded":"2024-09-11T01:21:27.552665+0000","last_fullsized":"2024-09-11T01:21:27.552665+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11748,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14776","reported_seq":23119,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.350482+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:30.350482+0000","last_peered":"2024-09-11T01:21:30.350482+0000","last_clean":"2024-09-11T01:21:30.350482+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:30.350482+0000","last_undegraded":"2024-09-11T01:21:30.350482+0000","last_fullsized":"2024-09-11T01:21:30.350482+0000","mapping_epoch":40,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22417891,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12643,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14897","reported_seq":15175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.570540+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:27.570540+0000","last_peered":"2024-09-11T01:21:27.570540+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:27.570540+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11261,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14655","reported_seq":16971,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.783924+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:30.783924+0000","last_peered":"2024-09-11T01:21:30.783924+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:30.783924+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22462947,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12424,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1110,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15034","reported_seq":15187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.770265+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:30.770265+0000","last_peered":"2024-09-11T01:21:30.770265+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:30.770265+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1381}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14864","reported_seq":23335,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.281781+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:30.281781+0000","last_peered":"2024-09-11T01:21:30.281781+0000","last_clean":"2024-09-11T01:21:30.281781+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:30.281781+0000","last_undegraded":"2024-09-11T01:21:30.281781+0000","last_fullsized":"2024-09-11T01:21:30.281781+0000","mapping_epoch":42,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15006","reported_seq":15291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.527476+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:27.527476+0000","last_peered":"2024-09-11T01:21:27.527476+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:27.527476+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23101440,"num_objects":5640,"num_object_clones":0,"num_object_copies":11280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11399,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14935","reported_seq":15235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.453907+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:27.453907+0000","last_peered":"2024-09-11T01:21:27.453907+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:27.453907+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11470,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14656","reported_seq":14896,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.576744+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:27.576744+0000","last_peered":"2024-09-11T01:21:27.576744+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:27.576744+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22455238,"num_objects":5484,"num_object_clones":0,"num_object_copies":10968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11313,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364593910,"num_objects":89022,"num_object_clones":0,"num_object_copies":178044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89022,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":196219,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9976,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930611200,"data_stored":930458536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621132,"kb_used_data":310396,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750708,"statfs":{"total":96636764160,"available":96000724992,"internally_reserved":0,"allocated":317845504,"data_stored":317563676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626876,"kb_used_data":266152,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93744964,"statfs":{"total":96636764160,"available":95994843136,"internally_reserved":0,"allocated":272539648,"data_stored":272217613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736836,"kb_used_data":335900,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93635004,"statfs":{"total":96636764160,"available":95882244096,"internally_reserved":0,"allocated":343961600,"data_stored":343679767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342650880,"data_stored":342614750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":271036416,"data_stored":270960084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316923904,"data_stored":316883702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:32.677 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14680", 'reported_seq': 14904, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.563128+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:27.563128+0000', 'last_peered': '2024-09-11T01:21:27.563128+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:27.563128+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'4600", 'ondisk_log_start': "39'4600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22765568, 'num_objects': 5558, 'num_object_clones': 0, 'num_object_copies': 11116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14680, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1510}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'14569", 'reported_seq': 14737, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:30.773418+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:30.773418+0000', 'last_peered': '2024-09-11T01:21:30.773418+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:30.773418+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'4500", 'ondisk_log_start': "39'4500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22429720, 'num_objects': 5477, 'num_object_clones': 0, 'num_object_copies': 10954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5477, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12630, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14655", 'reported_seq': 14922, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.476254+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:27.476254+0000', 'last_peered': '2024-09-11T01:21:27.476254+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:27.476254+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'4600", 'ondisk_log_start': "39'4600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22741475, 'num_objects': 5553, 'num_object_clones': 0, 'num_object_copies': 11106, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5553, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11143, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1504}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'15172", 'reported_seq': 15459, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.503428+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:27.503428+0000', 'last_peered': '2024-09-11T01:21:27.503428+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:27.503428+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'5100", 'ondisk_log_start': "39'5100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23102406, 'num_objects': 5642, 'num_object_clones': 0, 'num_object_copies': 11284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5642, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 11925, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1436}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'14897", 'reported_seq': 15175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.570540+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:27.570540+0000', 'last_peered': '2024-09-11T01:21:27.570540+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:27.570540+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23334912, 'num_objects': 5697, 'num_object_clones': 0, 'num_object_copies': 11394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11261, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1667}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'14655", 'reported_seq': 16971, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:30.783924+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:30.783924+0000', 'last_peered': '2024-09-11T01:21:30.783924+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:30.783924+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'4600", 'ondisk_log_start': "39'4600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22462947, 'num_objects': 5485, 'num_object_clones': 0, 'num_object_copies': 10970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2918, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5485, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12424, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1110, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2567}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15034", 'reported_seq': 15187, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:30.770265+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:30.770265+0000', 'last_peered': '2024-09-11T01:21:30.770265+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:30.770265+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22642688, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12671, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1381}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'15006", 'reported_seq': 15291, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.527476+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:27.527476+0000', 'last_peered': '2024-09-11T01:21:27.527476+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:27.527476+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23101440, 'num_objects': 5640, 'num_object_clones': 0, 'num_object_copies': 11280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11399, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'14935", 'reported_seq': 15235, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.453907+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:27.453907+0000', 'last_peered': '2024-09-11T01:21:27.453907+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:27.453907+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'4900", 'ondisk_log_start': "39'4900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23203840, 'num_objects': 5665, 'num_object_clones': 0, 'num_object_copies': 11330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11470, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1600}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:32.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14656", 'reported_seq': 14896, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.576744+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:27.576744+0000', 'last_peered': '2024-09-11T01:21:27.576744+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:27.576744+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'4600", 'ondisk_log_start': "39'4600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22455238, 'num_objects': 5484, 'num_object_clones': 0, 'num_object_copies': 10968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5484, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11313, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1382}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:32.680 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:33.418 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:33.418 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39658/178048 objects degraded (22.274%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":493,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89024,"data_bytes":364593929,"bytes_used":2032480256,"bytes_avail":287877812224,"bytes_total":289910292480,"degraded_objects":39658,"degraded_total":178048,"degraded_ratio":0.222737688713156,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":444},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:33.444 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:33.444 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:34.045 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:34.046 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:34.068 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-11T01:21:32.576382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364593929,"num_objects":89024,"num_object_clones":0,"num_object_copies":178048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89024,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":196221,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9977,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160859,"ondisk_log_size":160859,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1984844,"kb_used_data":912448,"kb_used_omap":12,"kb_used_meta":1072307,"kb_avail":281130676,"statfs":{"total":289910292480,"available":287877812224,"internally_reserved":0,"allocated":934346752,"data_stored":933461056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1098042375},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,7,6,7,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21880832,"num_objects":-5342,"num_object_clones":0,"num_object_copies":-10684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5342,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005269"},"pg_stats":[{"pgid":"2.f","version":"43'14680","reported_seq":14904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.563128+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:27.563128+0000","last_peered":"2024-09-11T01:21:27.563128+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:27.563128+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22765568,"num_objects":5558,"num_object_clones":0,"num_object_copies":11116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'14924","reported_seq":15005,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.789535+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:30.789535+0000","last_peered":"2024-09-11T01:21:30.789535+0000","last_clean":"2024-09-11T01:21:30.789535+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:30.789535+0000","last_undegraded":"2024-09-11T01:21:30.789535+0000","last_fullsized":"2024-09-11T01:21:30.789535+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22930374,"num_objects":5600,"num_object_clones":0,"num_object_copies":11200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5600,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13022,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14569","reported_seq":14737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.773418+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:30.773418+0000","last_peered":"2024-09-11T01:21:30.773418+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:30.773418+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4500","ondisk_log_start":"39'4500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22429720,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12630,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14819","reported_seq":14865,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.784374+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:30.784374+0000","last_peered":"2024-09-11T01:21:30.784374+0000","last_clean":"2024-09-11T01:21:30.784374+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:30.784374+0000","last_undegraded":"2024-09-11T01:21:30.784374+0000","last_fullsized":"2024-09-11T01:21:30.784374+0000","mapping_epoch":31,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704128,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14719","reported_seq":14841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.492972+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:27.492972+0000","last_peered":"2024-09-11T01:21:27.492972+0000","last_clean":"2024-09-11T01:21:27.492972+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:27.492972+0000","last_undegraded":"2024-09-11T01:21:27.492972+0000","last_fullsized":"2024-09-11T01:21:27.492972+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22766051,"num_objects":5559,"num_object_clones":0,"num_object_copies":11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5559,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11517,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14655","reported_seq":14922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.476254+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:27.476254+0000","last_peered":"2024-09-11T01:21:27.476254+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:27.476254+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22741475,"num_objects":5553,"num_object_clones":0,"num_object_copies":11106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11143,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1504},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15172","reported_seq":15459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.503428+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:27.503428+0000","last_peered":"2024-09-11T01:21:27.503428+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:27.503428+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23102406,"num_objects":5642,"num_object_clones":0,"num_object_copies":11284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5642,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11925,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1436},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14795","reported_seq":14915,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.552665+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:27.552665+0000","last_peered":"2024-09-11T01:21:27.552665+0000","last_clean":"2024-09-11T01:21:27.552665+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:27.552665+0000","last_undegraded":"2024-09-11T01:21:27.552665+0000","last_fullsized":"2024-09-11T01:21:27.552665+0000","mapping_epoch":31,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23056384,"num_objects":5629,"num_object_clones":0,"num_object_copies":11258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11748,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14776","reported_seq":23119,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.350482+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:30.350482+0000","last_peered":"2024-09-11T01:21:30.350482+0000","last_clean":"2024-09-11T01:21:30.350482+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:30.350482+0000","last_undegraded":"2024-09-11T01:21:30.350482+0000","last_fullsized":"2024-09-11T01:21:30.350482+0000","mapping_epoch":40,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22417891,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12643,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'14897","reported_seq":15175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.570540+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:27.570540+0000","last_peered":"2024-09-11T01:21:27.570540+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:27.570540+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23334912,"num_objects":5697,"num_object_clones":0,"num_object_copies":11394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11261,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14655","reported_seq":16971,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.783924+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:30.783924+0000","last_peered":"2024-09-11T01:21:30.783924+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:30.783924+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22462947,"num_objects":5485,"num_object_clones":0,"num_object_copies":10970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5485,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12424,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1110,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680146+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680146+0000","last_peered":"2024-09-11T01:20:45.680146+0000","last_clean":"2024-09-11T01:20:45.680146+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680146+0000","last_undegraded":"2024-09-11T01:20:45.680146+0000","last_fullsized":"2024-09-11T01:20:45.680146+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15034","reported_seq":15187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.770265+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:30.770265+0000","last_peered":"2024-09-11T01:21:30.770265+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:30.770265+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22642688,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12671,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1381}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'14864","reported_seq":23335,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.281781+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:30.281781+0000","last_peered":"2024-09-11T01:21:30.281781+0000","last_clean":"2024-09-11T01:21:30.281781+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:30.281781+0000","last_undegraded":"2024-09-11T01:21:30.281781+0000","last_fullsized":"2024-09-11T01:21:30.281781+0000","mapping_epoch":42,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22478848,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11554,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680241+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680241+0000","last_peered":"2024-09-11T01:20:45.680241+0000","last_clean":"2024-09-11T01:20:45.680241+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680241+0000","last_undegraded":"2024-09-11T01:20:45.680241+0000","last_fullsized":"2024-09-11T01:20:45.680241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15006","reported_seq":15291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.527476+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:27.527476+0000","last_peered":"2024-09-11T01:21:27.527476+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:27.527476+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23101440,"num_objects":5640,"num_object_clones":0,"num_object_copies":11280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11399,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'14935","reported_seq":15235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.453907+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:27.453907+0000","last_peered":"2024-09-11T01:21:27.453907+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:27.453907+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23203840,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11470,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680367+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680367+0000","last_peered":"2024-09-11T01:20:45.680367+0000","last_clean":"2024-09-11T01:20:45.680367+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680367+0000","last_undegraded":"2024-09-11T01:20:45.680367+0000","last_fullsized":"2024-09-11T01:20:45.680367+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14656","reported_seq":14896,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.576744+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:27.576744+0000","last_peered":"2024-09-11T01:21:27.576744+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:27.576744+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4600","ondisk_log_start":"39'4600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22455238,"num_objects":5484,"num_object_clones":0,"num_object_copies":10968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5484,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11313,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364593910,"num_objects":89022,"num_object_clones":0,"num_object_copies":178044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89022,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":196219,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9976,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":930611200,"data_stored":930458536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621132,"kb_used_data":310396,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750708,"statfs":{"total":96636764160,"available":96000724992,"internally_reserved":0,"allocated":317845504,"data_stored":317563676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790027,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626876,"kb_used_data":266152,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93744964,"statfs":{"total":96636764160,"available":95994843136,"internally_reserved":0,"allocated":272539648,"data_stored":272217613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,1,3,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736836,"kb_used_data":335900,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93635004,"statfs":{"total":96636764160,"available":95882244096,"internally_reserved":0,"allocated":343961600,"data_stored":343679767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":410513066},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":342650880,"data_stored":342614750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":271036416,"data_stored":270960084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":316923904,"data_stored":316883702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:37.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:37.628 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:37.630 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:37.651 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-11T01:21:36.577908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355078921,"num_objects":86701,"num_object_clones":0,"num_object_copies":173402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86701,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":198544,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10068,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1937164,"kb_used_data":893896,"kb_used_omap":12,"kb_used_meta":1043187,"kb_avail":281178356,"statfs":{"total":289910292480,"available":287926636544,"internally_reserved":0,"allocated":915349504,"data_stored":914463808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1068223495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,7,9,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23089152,"num_objects":-5637,"num_object_clones":0,"num_object_copies":-11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5637,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005223"},"pg_stats":[{"pgid":"2.f","version":"43'14831","reported_seq":15055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.565781+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:32.565781+0000","last_peered":"2024-09-11T01:21:32.565781+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:32.565781+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14831,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15088","reported_seq":15169,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.813796+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:35.813796+0000","last_peered":"2024-09-11T01:21:35.813796+0000","last_clean":"2024-09-11T01:21:35.813796+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:35.813796+0000","last_undegraded":"2024-09-11T01:21:35.813796+0000","last_fullsized":"2024-09-11T01:21:35.813796+0000","mapping_epoch":31,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13186,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14717","reported_seq":14882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.727302+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:35.727302+0000","last_peered":"2024-09-11T01:21:35.727302+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:35.727302+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823512,"num_objects":5329,"num_object_clones":0,"num_object_copies":10658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12778,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1366}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14977","reported_seq":15023,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.791482+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:35.791482+0000","last_peered":"2024-09-11T01:21:35.791482+0000","last_clean":"2024-09-11T01:21:35.791482+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:35.791482+0000","last_undegraded":"2024-09-11T01:21:35.791482+0000","last_fullsized":"2024-09-11T01:21:35.791482+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14977,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14866","reported_seq":14988,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.548319+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:32.548319+0000","last_peered":"2024-09-11T01:21:32.548319+0000","last_clean":"2024-09-11T01:21:32.548319+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:32.548319+0000","last_undegraded":"2024-09-11T01:21:32.548319+0000","last_fullsized":"2024-09-11T01:21:32.548319+0000","mapping_epoch":31,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22163939,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11664,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14786","reported_seq":15053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.504591+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:32.504591+0000","last_peered":"2024-09-11T01:21:32.504591+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:32.504591+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22204899,"num_objects":5422,"num_object_clones":0,"num_object_copies":10844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5422,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11274,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15324","reported_seq":15612,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.568120+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:32.568120+0000","last_peered":"2024-09-11T01:21:32.568120+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:32.568120+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22479814,"num_objects":5490,"num_object_clones":0,"num_object_copies":10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5490,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12077,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1284},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14937","reported_seq":15057,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.526192+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:32.526192+0000","last_peered":"2024-09-11T01:21:32.526192+0000","last_clean":"2024-09-11T01:21:32.526192+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:32.526192+0000","last_undegraded":"2024-09-11T01:21:32.526192+0000","last_fullsized":"2024-09-11T01:21:32.526192+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22474752,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14906","reported_seq":23248,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.435705+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:35.435705+0000","last_peered":"2024-09-11T01:21:35.435705+0000","last_clean":"2024-09-11T01:21:35.435705+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:35.435705+0000","last_undegraded":"2024-09-11T01:21:35.435705+0000","last_fullsized":"2024-09-11T01:21:35.435705+0000","mapping_epoch":40,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21885411,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12773,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15013","reported_seq":15293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.536988+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:32.536988+0000","last_peered":"2024-09-11T01:21:32.536988+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:32.536988+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1551}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14803","reported_seq":17301,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.795206+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:35.795206+0000","last_peered":"2024-09-11T01:21:35.795206+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:35.795206+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856739,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12572,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15161","reported_seq":15315,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.738620+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:35.738620+0000","last_peered":"2024-09-11T01:21:35.738620+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:35.738620+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22122496,"num_objects":5401,"num_object_clones":0,"num_object_copies":10802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15022","reported_seq":23493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.368327+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:35.368327+0000","last_peered":"2024-09-11T01:21:35.368327+0000","last_clean":"2024-09-11T01:21:35.368327+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:35.368327+0000","last_undegraded":"2024-09-11T01:21:35.368327+0000","last_fullsized":"2024-09-11T01:21:35.368327+0000","mapping_epoch":42,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15169","reported_seq":15451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.607529+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:32.607529+0000","last_peered":"2024-09-11T01:21:32.607529+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:32.607529+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11562,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15090","reported_seq":15390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.549858+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:32.549858+0000","last_peered":"2024-09-11T01:21:32.549858+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:32.549858+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11625,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14789","reported_seq":15029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.565517+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:32.565517+0000","last_peered":"2024-09-11T01:21:32.565517+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:32.565517+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21910470,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11446,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":355078902,"num_objects":86699,"num_object_clones":0,"num_object_copies":173398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86699,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":198542,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10067,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":911601664,"data_stored":911449000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604068,"kb_used_data":305812,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93767772,"statfs":{"total":96636764160,"available":96018198528,"internally_reserved":0,"allocated":313151488,"data_stored":312869660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790028,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611772,"kb_used_data":259352,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93760068,"statfs":{"total":96636764160,"available":96010309632,"internally_reserved":0,"allocated":265576448,"data_stored":265254413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721324,"kb_used_data":328732,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93650516,"statfs":{"total":96636764160,"available":95898128384,"internally_reserved":0,"allocated":336621568,"data_stored":336339735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335409152,"data_stored":335373022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":264028160,"data_stored":263951828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312164352,"data_stored":312124150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:37.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:38.259 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:38.261 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:38.280 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-11T01:21:36.577908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":355078921,"num_objects":86701,"num_object_clones":0,"num_object_copies":173402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86701,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":198544,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10068,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1937164,"kb_used_data":893896,"kb_used_omap":12,"kb_used_meta":1043187,"kb_avail":281178356,"statfs":{"total":289910292480,"available":287926636544,"internally_reserved":0,"allocated":915349504,"data_stored":914463808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1068223495},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,7,9,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23089152,"num_objects":-5637,"num_object_clones":0,"num_object_copies":-11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-258,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5637,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005223"},"pg_stats":[{"pgid":"2.f","version":"43'14831","reported_seq":15055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.565781+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:32.565781+0000","last_peered":"2024-09-11T01:21:32.565781+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:32.565781+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14831,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15088","reported_seq":15169,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.813796+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:35.813796+0000","last_peered":"2024-09-11T01:21:35.813796+0000","last_clean":"2024-09-11T01:21:35.813796+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:35.813796+0000","last_undegraded":"2024-09-11T01:21:35.813796+0000","last_fullsized":"2024-09-11T01:21:35.813796+0000","mapping_epoch":31,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13186,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14717","reported_seq":14882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.727302+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:35.727302+0000","last_peered":"2024-09-11T01:21:35.727302+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:35.727302+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823512,"num_objects":5329,"num_object_clones":0,"num_object_copies":10658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12778,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1366}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14977","reported_seq":15023,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.791482+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:35.791482+0000","last_peered":"2024-09-11T01:21:35.791482+0000","last_clean":"2024-09-11T01:21:35.791482+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:35.791482+0000","last_undegraded":"2024-09-11T01:21:35.791482+0000","last_fullsized":"2024-09-11T01:21:35.791482+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14977,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14866","reported_seq":14988,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.548319+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:32.548319+0000","last_peered":"2024-09-11T01:21:32.548319+0000","last_clean":"2024-09-11T01:21:32.548319+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:32.548319+0000","last_undegraded":"2024-09-11T01:21:32.548319+0000","last_fullsized":"2024-09-11T01:21:32.548319+0000","mapping_epoch":31,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22163939,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11664,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14786","reported_seq":15053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.504591+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:32.504591+0000","last_peered":"2024-09-11T01:21:32.504591+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:32.504591+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22204899,"num_objects":5422,"num_object_clones":0,"num_object_copies":10844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5422,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11274,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1373},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15324","reported_seq":15612,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.568120+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:32.568120+0000","last_peered":"2024-09-11T01:21:32.568120+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:32.568120+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22479814,"num_objects":5490,"num_object_clones":0,"num_object_copies":10980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5490,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12077,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1284},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'14937","reported_seq":15057,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.526192+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:32.526192+0000","last_peered":"2024-09-11T01:21:32.526192+0000","last_clean":"2024-09-11T01:21:32.526192+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:32.526192+0000","last_undegraded":"2024-09-11T01:21:32.526192+0000","last_fullsized":"2024-09-11T01:21:32.526192+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22474752,"num_objects":5487,"num_object_clones":0,"num_object_copies":10974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11890,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14906","reported_seq":23248,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.435705+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:35.435705+0000","last_peered":"2024-09-11T01:21:35.435705+0000","last_clean":"2024-09-11T01:21:35.435705+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:35.435705+0000","last_undegraded":"2024-09-11T01:21:35.435705+0000","last_fullsized":"2024-09-11T01:21:35.435705+0000","mapping_epoch":40,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21885411,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12773,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15013","reported_seq":15293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.536988+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:32.536988+0000","last_peered":"2024-09-11T01:21:32.536988+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:32.536988+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22859776,"num_objects":5581,"num_object_clones":0,"num_object_copies":11162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11377,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1551}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14803","reported_seq":17301,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.795206+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:35.795206+0000","last_peered":"2024-09-11T01:21:35.795206+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:35.795206+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856739,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12572,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15161","reported_seq":15315,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.738620+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:35.738620+0000","last_peered":"2024-09-11T01:21:35.738620+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:35.738620+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22122496,"num_objects":5401,"num_object_clones":0,"num_object_copies":10802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15022","reported_seq":23493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.368327+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:35.368327+0000","last_peered":"2024-09-11T01:21:35.368327+0000","last_clean":"2024-09-11T01:21:35.368327+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:35.368327+0000","last_undegraded":"2024-09-11T01:21:35.368327+0000","last_fullsized":"2024-09-11T01:21:35.368327+0000","mapping_epoch":42,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15169","reported_seq":15451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.607529+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:32.607529+0000","last_peered":"2024-09-11T01:21:32.607529+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:32.607529+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11562,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15090","reported_seq":15390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.549858+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:32.549858+0000","last_peered":"2024-09-11T01:21:32.549858+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:32.549858+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22568960,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11625,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1445}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14789","reported_seq":15029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.565517+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:32.565517+0000","last_peered":"2024-09-11T01:21:32.565517+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:32.565517+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21910470,"num_objects":5351,"num_object_clones":0,"num_object_copies":10702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11446,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":355078902,"num_objects":86699,"num_object_clones":0,"num_object_copies":173398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86699,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":198542,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10067,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":911601664,"data_stored":911449000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160779,"ondisk_log_size":160779,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604068,"kb_used_data":305812,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93767772,"statfs":{"total":96636764160,"available":96018198528,"internally_reserved":0,"allocated":313151488,"data_stored":312869660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790028,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611772,"kb_used_data":259352,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93760068,"statfs":{"total":96636764160,"available":96010309632,"internally_reserved":0,"allocated":265576448,"data_stored":265254413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721324,"kb_used_data":328732,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93650516,"statfs":{"total":96636764160,"available":95898128384,"internally_reserved":0,"allocated":336621568,"data_stored":336339735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,6,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":335409152,"data_stored":335373022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":264028160,"data_stored":263951828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312164352,"data_stored":312124150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:38.281 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:38.281 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14831", 'reported_seq': 15055, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.565781+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:32.565781+0000', 'last_peered': '2024-09-11T01:21:32.565781+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:32.565781+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22147072, 'num_objects': 5407, 'num_object_clones': 0, 'num_object_copies': 10814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14831, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1359}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'14717", 'reported_seq': 14882, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:35.727302+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:35.727302+0000', 'last_peered': '2024-09-11T01:21:35.727302+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:35.727302+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'4700", 'ondisk_log_start': "39'4700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21823512, 'num_objects': 5329, 'num_object_clones': 0, 'num_object_copies': 10658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5329, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12778, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1366}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14786", 'reported_seq': 15053, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.504591+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:32.504591+0000', 'last_peered': '2024-09-11T01:21:32.504591+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:32.504591+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'4700", 'ondisk_log_start': "39'4700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22204899, 'num_objects': 5422, 'num_object_clones': 0, 'num_object_copies': 10844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5422, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11274, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1373}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'15324", 'reported_seq': 15612, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.568120+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:32.568120+0000', 'last_peered': '2024-09-11T01:21:32.568120+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:32.568120+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'5300", 'ondisk_log_start': "39'5300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22479814, 'num_objects': 5490, 'num_object_clones': 0, 'num_object_copies': 10980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5490, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12077, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1284}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'15013", 'reported_seq': 15293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.536988+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:32.536988+0000', 'last_peered': '2024-09-11T01:21:32.536988+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:32.536988+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22859776, 'num_objects': 5581, 'num_object_clones': 0, 'num_object_copies': 11162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11377, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1551}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:38.282 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'14803", 'reported_seq': 17301, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:35.795206+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:35.795206+0000', 'last_peered': '2024-09-11T01:21:35.795206+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:35.795206+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21856739, 'num_objects': 5337, 'num_object_clones': 0, 'num_object_copies': 10674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2827, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5337, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12572, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1201, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2510}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15161", 'reported_seq': 15315, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:35.738620+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:35.738620+0000', 'last_peered': '2024-09-11T01:21:35.738620+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:35.738620+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5100", 'ondisk_log_start': "39'5100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22122496, 'num_objects': 5401, 'num_object_clones': 0, 'num_object_copies': 10802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5401, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12798, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1254}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'15169", 'reported_seq': 15451, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.607529+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:32.607529+0000', 'last_peered': '2024-09-11T01:21:32.607529+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:32.607529+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'5100", 'ondisk_log_start': "39'5100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22433792, 'num_objects': 5477, 'num_object_clones': 0, 'num_object_copies': 10954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11562, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1360}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'15090", 'reported_seq': 15390, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.549858+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:32.549858+0000', 'last_peered': '2024-09-11T01:21:32.549858+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:32.549858+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22568960, 'num_objects': 5510, 'num_object_clones': 0, 'num_object_copies': 11020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5510, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11625, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1445}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:38.283 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14789", 'reported_seq': 15029, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:32.565517+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:32.565517+0000', 'last_peered': '2024-09-11T01:21:32.565517+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:32.565517+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'4700", 'ondisk_log_start': "39'4700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21910470, 'num_objects': 5351, 'num_object_clones': 0, 'num_object_copies': 10702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5351, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11446, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1249}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:38.283 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:39.030 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:39.030 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39567/170792 objects degraded (23.167%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":499,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":85396,"data_bytes":349733641,"bytes_used":1980801024,"bytes_avail":287929491456,"bytes_total":289910292480,"degraded_objects":39567,"degraded_total":170792,"degraded_ratio":0.23166775961403341,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:39.054 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:39.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:39.709 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:39.709 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:39.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-11T01:21:38.578857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":349733641,"num_objects":85396,"num_object_clones":0,"num_object_copies":170792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85396,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":199849,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10068,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160887,"ondisk_log_size":160887,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1934376,"kb_used_data":886948,"kb_used_omap":12,"kb_used_meta":1047347,"kb_avail":281181144,"statfs":{"total":289910292480,"available":287929491456,"internally_reserved":0,"allocated":908234752,"data_stored":907349056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1072483335},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,8,4,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24412160,"num_objects":-5960,"num_object_clones":0,"num_object_copies":-11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5960,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005246"},"pg_stats":[{"pgid":"2.f","version":"43'14985","reported_seq":15209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.722667+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:37.722667+0000","last_peered":"2024-09-11T01:21:37.722667+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:37.722667+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15088","reported_seq":15169,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.813796+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:35.813796+0000","last_peered":"2024-09-11T01:21:35.813796+0000","last_clean":"2024-09-11T01:21:35.813796+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:35.813796+0000","last_undegraded":"2024-09-11T01:21:35.813796+0000","last_fullsized":"2024-09-11T01:21:35.813796+0000","mapping_epoch":31,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13186,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14717","reported_seq":14882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.727302+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:35.727302+0000","last_peered":"2024-09-11T01:21:35.727302+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:35.727302+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4700","ondisk_log_start":"39'4700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21823512,"num_objects":5329,"num_object_clones":0,"num_object_copies":10658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5329,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12778,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1366}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'14977","reported_seq":15023,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.791482+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:35.791482+0000","last_peered":"2024-09-11T01:21:35.791482+0000","last_clean":"2024-09-11T01:21:35.791482+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:35.791482+0000","last_undegraded":"2024-09-11T01:21:35.791482+0000","last_fullsized":"2024-09-11T01:21:35.791482+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22056960,"num_objects":5385,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14977,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14995","reported_seq":15117,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.677864+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:37.677864+0000","last_peered":"2024-09-11T01:21:37.677864+0000","last_clean":"2024-09-11T01:21:37.677864+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:37.677864+0000","last_undegraded":"2024-09-11T01:21:37.677864+0000","last_fullsized":"2024-09-11T01:21:37.677864+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21635555,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11793,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14941","reported_seq":15206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.658891+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:37.658891+0000","last_peered":"2024-09-11T01:21:37.658891+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:37.658891+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21570019,"num_objects":5267,"num_object_clones":0,"num_object_copies":10534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11429,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15457","reported_seq":15745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.671303+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:37.671303+0000","last_peered":"2024-09-11T01:21:37.671303+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:37.671303+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21935046,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12210,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15081","reported_seq":15200,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.622889+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:37.622889+0000","last_peered":"2024-09-11T01:21:37.622889+0000","last_clean":"2024-09-11T01:21:37.622889+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:37.622889+0000","last_undegraded":"2024-09-11T01:21:37.622889+0000","last_fullsized":"2024-09-11T01:21:37.622889+0000","mapping_epoch":31,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12034,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'14906","reported_seq":23248,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.435705+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:35.435705+0000","last_peered":"2024-09-11T01:21:35.435705+0000","last_clean":"2024-09-11T01:21:35.435705+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:35.435705+0000","last_undegraded":"2024-09-11T01:21:35.435705+0000","last_fullsized":"2024-09-11T01:21:35.435705+0000","mapping_epoch":40,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21885411,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12773,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15182","reported_seq":15462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.655771+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:37.655771+0000","last_peered":"2024-09-11T01:21:37.655771+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:37.655771+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11546,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14803","reported_seq":17301,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.795206+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:35.795206+0000","last_peered":"2024-09-11T01:21:35.795206+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:35.795206+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21856739,"num_objects":5337,"num_object_clones":0,"num_object_copies":10674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2827,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12572,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1201,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2510}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15161","reported_seq":15315,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.738620+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:35.738620+0000","last_peered":"2024-09-11T01:21:35.738620+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:35.738620+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22122496,"num_objects":5401,"num_object_clones":0,"num_object_copies":10802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12798,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1254}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15022","reported_seq":23493,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.368327+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:35.368327+0000","last_peered":"2024-09-11T01:21:35.368327+0000","last_clean":"2024-09-11T01:21:35.368327+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:35.368327+0000","last_undegraded":"2024-09-11T01:21:35.368327+0000","last_fullsized":"2024-09-11T01:21:35.368327+0000","mapping_epoch":42,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21831680,"num_objects":5330,"num_object_clones":0,"num_object_copies":10660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11712,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15299","reported_seq":15584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.728661+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:37.728661+0000","last_peered":"2024-09-11T01:21:37.728661+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:37.728661+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1230}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15235","reported_seq":15535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.661461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:37.661461+0000","last_peered":"2024-09-11T01:21:37.661461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:37.661461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14935","reported_seq":15175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.648831+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:37.648831+0000","last_peered":"2024-09-11T01:21:37.648831+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:37.648831+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21312454,"num_objects":5205,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11592,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":349733622,"num_objects":85394,"num_object_clones":0,"num_object_copies":170788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39567,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85394,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":199847,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10067,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":904388608,"data_stored":904235944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604068,"kb_used_data":305812,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93767772,"statfs":{"total":96636764160,"available":96018198528,"internally_reserved":0,"allocated":313151488,"data_stored":312869660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790028,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611772,"kb_used_data":259352,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93760068,"statfs":{"total":96636764160,"available":96010309632,"internally_reserved":0,"allocated":265576448,"data_stored":265254413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718536,"kb_used_data":321784,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93653304,"statfs":{"total":96636764160,"available":95900983296,"internally_reserved":0,"allocated":329506816,"data_stored":329224983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328196096,"data_stored":328159966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":264028160,"data_stored":263951828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312164352,"data_stored":312124150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:42.739 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:43.333 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:43.335 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:43.360 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-11T01:21:42.580198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345617161,"num_objects":84391,"num_object_clones":0,"num_object_copies":168782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84391,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":200854,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10161,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1931548,"kb_used_data":875808,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281183972,"statfs":{"total":289910292480,"available":287932387328,"internally_reserved":0,"allocated":896827392,"data_stored":895941696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1081003015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,9,6,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21954560,"num_objects":-5360,"num_object_clones":0,"num_object_copies":-10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5360,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004734"},"pg_stats":[{"pgid":"2.f","version":"43'14985","reported_seq":15209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.722667+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:37.722667+0000","last_peered":"2024-09-11T01:21:37.722667+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:37.722667+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15235","reported_seq":15316,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.819559+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:40.819559+0000","last_peered":"2024-09-11T01:21:40.819559+0000","last_clean":"2024-09-11T01:21:40.819559+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:40.819559+0000","last_undegraded":"2024-09-11T01:21:40.819559+0000","last_fullsized":"2024-09-11T01:21:40.819559+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21656518,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13333,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14870","reported_seq":15039,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.699553+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:40.699553+0000","last_peered":"2024-09-11T01:21:40.699553+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:40.699553+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21196824,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12931,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1213}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15117","reported_seq":15163,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.836687+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:40.836687+0000","last_peered":"2024-09-11T01:21:40.836687+0000","last_clean":"2024-09-11T01:21:40.836687+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:40.836687+0000","last_undegraded":"2024-09-11T01:21:40.836687+0000","last_fullsized":"2024-09-11T01:21:40.836687+0000","mapping_epoch":31,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15117,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14995","reported_seq":15117,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.677864+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:37.677864+0000","last_peered":"2024-09-11T01:21:37.677864+0000","last_clean":"2024-09-11T01:21:37.677864+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:37.677864+0000","last_undegraded":"2024-09-11T01:21:37.677864+0000","last_fullsized":"2024-09-11T01:21:37.677864+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21635555,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11793,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14941","reported_seq":15206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.658891+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:37.658891+0000","last_peered":"2024-09-11T01:21:37.658891+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:37.658891+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21570019,"num_objects":5267,"num_object_clones":0,"num_object_copies":10534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11429,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15457","reported_seq":15745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.671303+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:37.671303+0000","last_peered":"2024-09-11T01:21:37.671303+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:37.671303+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21935046,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12210,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15081","reported_seq":15200,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.622889+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:37.622889+0000","last_peered":"2024-09-11T01:21:37.622889+0000","last_clean":"2024-09-11T01:21:37.622889+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:37.622889+0000","last_undegraded":"2024-09-11T01:21:37.622889+0000","last_fullsized":"2024-09-11T01:21:37.622889+0000","mapping_epoch":31,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12034,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15049","reported_seq":23392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.465862+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:40.465862+0000","last_peered":"2024-09-11T01:21:40.465862+0000","last_clean":"2024-09-11T01:21:40.465862+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:40.465862+0000","last_undegraded":"2024-09-11T01:21:40.465862+0000","last_fullsized":"2024-09-11T01:21:40.465862+0000","mapping_epoch":40,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21299683,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12916,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15182","reported_seq":15462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.655771+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:37.655771+0000","last_peered":"2024-09-11T01:21:37.655771+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:37.655771+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11546,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14925","reported_seq":17609,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.790991+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:40.790991+0000","last_peered":"2024-09-11T01:21:40.790991+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:40.790991+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21357027,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12694,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1294,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15307","reported_seq":15461,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.782184+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:40.782184+0000","last_peered":"2024-09-11T01:21:40.782184+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:40.782184+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21524480,"num_objects":5255,"num_object_clones":0,"num_object_copies":10510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15176","reported_seq":23647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.459564+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:40.459564+0000","last_peered":"2024-09-11T01:21:40.459564+0000","last_clean":"2024-09-11T01:21:40.459564+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:40.459564+0000","last_undegraded":"2024-09-11T01:21:40.459564+0000","last_fullsized":"2024-09-11T01:21:40.459564+0000","mapping_epoch":42,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11866,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15299","reported_seq":15584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.728661+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:37.728661+0000","last_peered":"2024-09-11T01:21:37.728661+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:37.728661+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1230}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15235","reported_seq":15535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.661461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:37.661461+0000","last_peered":"2024-09-11T01:21:37.661461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:37.661461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14935","reported_seq":15175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.648831+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:37.648831+0000","last_peered":"2024-09-11T01:21:37.648831+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:37.648831+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21312454,"num_objects":5205,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11592,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345617142,"num_objects":84389,"num_object_clones":0,"num_object_copies":168778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84389,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":200852,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10160,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":892841984,"data_stored":892689320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603808,"kb_used_data":301384,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93768032,"statfs":{"total":96636764160,"available":96018464768,"internally_reserved":0,"allocated":308617216,"data_stored":308335388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790029,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609204,"kb_used_data":252640,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93762636,"statfs":{"total":96636764160,"available":96012939264,"internally_reserved":0,"allocated":258703360,"data_stored":258381325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718536,"kb_used_data":321784,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93653304,"statfs":{"total":96636764160,"available":95900983296,"internally_reserved":0,"allocated":329506816,"data_stored":329224983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328196096,"data_stored":328159966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":256946176,"data_stored":256869844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307699712,"data_stored":307659510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:43.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:43.935 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:43.936 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:43.959 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-11T01:21:42.580198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345617161,"num_objects":84391,"num_object_clones":0,"num_object_copies":168782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84391,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":200854,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10161,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1931548,"kb_used_data":875808,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281183972,"statfs":{"total":289910292480,"available":287932387328,"internally_reserved":0,"allocated":896827392,"data_stored":895941696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1081003015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,9,6,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21954560,"num_objects":-5360,"num_object_clones":0,"num_object_copies":-10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5360,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004734"},"pg_stats":[{"pgid":"2.f","version":"43'14985","reported_seq":15209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.722667+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:37.722667+0000","last_peered":"2024-09-11T01:21:37.722667+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:37.722667+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21516288,"num_objects":5253,"num_object_clones":0,"num_object_copies":10506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1205}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15235","reported_seq":15316,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.819559+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:40.819559+0000","last_peered":"2024-09-11T01:21:40.819559+0000","last_clean":"2024-09-11T01:21:40.819559+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:40.819559+0000","last_undegraded":"2024-09-11T01:21:40.819559+0000","last_fullsized":"2024-09-11T01:21:40.819559+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21656518,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13333,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14870","reported_seq":15039,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.699553+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:40.699553+0000","last_peered":"2024-09-11T01:21:40.699553+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:40.699553+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21196824,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12931,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1213}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15117","reported_seq":15163,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.836687+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:40.836687+0000","last_peered":"2024-09-11T01:21:40.836687+0000","last_clean":"2024-09-11T01:21:40.836687+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:40.836687+0000","last_undegraded":"2024-09-11T01:21:40.836687+0000","last_fullsized":"2024-09-11T01:21:40.836687+0000","mapping_epoch":31,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15117,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'14995","reported_seq":15117,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.677864+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:37.677864+0000","last_peered":"2024-09-11T01:21:37.677864+0000","last_clean":"2024-09-11T01:21:37.677864+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:37.677864+0000","last_undegraded":"2024-09-11T01:21:37.677864+0000","last_fullsized":"2024-09-11T01:21:37.677864+0000","mapping_epoch":31,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21635555,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11793,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'14941","reported_seq":15206,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.658891+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:37.658891+0000","last_peered":"2024-09-11T01:21:37.658891+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:37.658891+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21570019,"num_objects":5267,"num_object_clones":0,"num_object_copies":10534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11429,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1218},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15457","reported_seq":15745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.671303+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:37.671303+0000","last_peered":"2024-09-11T01:21:37.671303+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:37.671303+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21935046,"num_objects":5357,"num_object_clones":0,"num_object_copies":10714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5357,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12210,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15081","reported_seq":15200,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.622889+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:37.622889+0000","last_peered":"2024-09-11T01:21:37.622889+0000","last_clean":"2024-09-11T01:21:37.622889+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:37.622889+0000","last_undegraded":"2024-09-11T01:21:37.622889+0000","last_fullsized":"2024-09-11T01:21:37.622889+0000","mapping_epoch":31,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21884928,"num_objects":5343,"num_object_clones":0,"num_object_copies":10686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12034,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15049","reported_seq":23392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.465862+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:40.465862+0000","last_peered":"2024-09-11T01:21:40.465862+0000","last_clean":"2024-09-11T01:21:40.465862+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:40.465862+0000","last_undegraded":"2024-09-11T01:21:40.465862+0000","last_fullsized":"2024-09-11T01:21:40.465862+0000","mapping_epoch":40,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21299683,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12916,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15182","reported_seq":15462,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.655771+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:37.655771+0000","last_peered":"2024-09-11T01:21:37.655771+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:37.655771+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22167552,"num_objects":5412,"num_object_clones":0,"num_object_copies":10824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11546,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14925","reported_seq":17609,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.790991+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:40.790991+0000","last_peered":"2024-09-11T01:21:40.790991+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:40.790991+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21357027,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12694,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1294,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15307","reported_seq":15461,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.782184+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:40.782184+0000","last_peered":"2024-09-11T01:21:40.782184+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:40.782184+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21524480,"num_objects":5255,"num_object_clones":0,"num_object_copies":10510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15176","reported_seq":23647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.459564+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:40.459564+0000","last_peered":"2024-09-11T01:21:40.459564+0000","last_clean":"2024-09-11T01:21:40.459564+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:40.459564+0000","last_undegraded":"2024-09-11T01:21:40.459564+0000","last_fullsized":"2024-09-11T01:21:40.459564+0000","mapping_epoch":42,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11866,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15299","reported_seq":15584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.728661+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:37.728661+0000","last_peered":"2024-09-11T01:21:37.728661+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:37.728661+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1230}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15235","reported_seq":15535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.661461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:37.661461+0000","last_peered":"2024-09-11T01:21:37.661461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:37.661461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21975040,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1300}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'14935","reported_seq":15175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.648831+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:37.648831+0000","last_peered":"2024-09-11T01:21:37.648831+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:37.648831+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21312454,"num_objects":5205,"num_object_clones":0,"num_object_copies":10410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11592,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1103},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345617142,"num_objects":84389,"num_object_clones":0,"num_object_copies":168778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84389,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":200852,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10160,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":892841984,"data_stored":892689320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603808,"kb_used_data":301384,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93768032,"statfs":{"total":96636764160,"available":96018464768,"internally_reserved":0,"allocated":308617216,"data_stored":308335388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790029,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609204,"kb_used_data":252640,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93762636,"statfs":{"total":96636764160,"available":96012939264,"internally_reserved":0,"allocated":258703360,"data_stored":258381325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718536,"kb_used_data":321784,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93653304,"statfs":{"total":96636764160,"available":95900983296,"internally_reserved":0,"allocated":329506816,"data_stored":329224983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":328196096,"data_stored":328159966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":256946176,"data_stored":256869844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307699712,"data_stored":307659510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:43.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:43.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'14985", 'reported_seq': 15209, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.722667+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:37.722667+0000', 'last_peered': '2024-09-11T01:21:37.722667+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:37.722667+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'4900", 'ondisk_log_start': "39'4900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21516288, 'num_objects': 5253, 'num_object_clones': 0, 'num_object_copies': 10506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14985, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1205}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'14870", 'reported_seq': 15039, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:40.699553+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:40.699553+0000', 'last_peered': '2024-09-11T01:21:40.699553+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:40.699553+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'4800", 'ondisk_log_start': "39'4800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21196824, 'num_objects': 5176, 'num_object_clones': 0, 'num_object_copies': 10352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5176, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12931, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1213}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'14941", 'reported_seq': 15206, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.658891+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:37.658891+0000', 'last_peered': '2024-09-11T01:21:37.658891+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:37.658891+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'4900", 'ondisk_log_start': "39'4900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21570019, 'num_objects': 5267, 'num_object_clones': 0, 'num_object_copies': 10534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5267, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11429, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1218}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'15457", 'reported_seq': 15745, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.671303+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:37.671303+0000', 'last_peered': '2024-09-11T01:21:37.671303+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:37.671303+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'5400", 'ondisk_log_start': "39'5400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21935046, 'num_objects': 5357, 'num_object_clones': 0, 'num_object_copies': 10714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5357, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12210, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1151}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'15182", 'reported_seq': 15462, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.655771+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:37.655771+0000', 'last_peered': '2024-09-11T01:21:37.655771+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:37.655771+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'5100", 'ondisk_log_start': "39'5100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22167552, 'num_objects': 5412, 'num_object_clones': 0, 'num_object_copies': 10824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5412, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11546, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1382}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:43.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'14925", 'reported_seq': 17609, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:40.790991+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:40.790991+0000', 'last_peered': '2024-09-11T01:21:40.790991+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:40.790991+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'4900", 'ondisk_log_start': "39'4900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21357027, 'num_objects': 5215, 'num_object_clones': 0, 'num_object_copies': 10430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2734, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5215, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12694, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1294, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2481}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15307", 'reported_seq': 15461, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:40.782184+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:40.782184+0000', 'last_peered': '2024-09-11T01:21:40.782184+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:40.782184+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5300", 'ondisk_log_start': "39'5300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21524480, 'num_objects': 5255, 'num_object_clones': 0, 'num_object_copies': 10510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12944, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1108}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'15299", 'reported_seq': 15584, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.728661+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:37.728661+0000', 'last_peered': '2024-09-11T01:21:37.728661+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:37.728661+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'5200", 'ondisk_log_start': "39'5200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21901312, 'num_objects': 5347, 'num_object_clones': 0, 'num_object_copies': 10694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11692, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1230}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'15235", 'reported_seq': 15535, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.661461+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:37.661461+0000', 'last_peered': '2024-09-11T01:21:37.661461+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:37.661461+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'5200", 'ondisk_log_start': "39'5200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21975040, 'num_objects': 5365, 'num_object_clones': 0, 'num_object_copies': 10730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5365, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11770, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1300}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:43.962 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'14935", 'reported_seq': 15175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.648831+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:37.648831+0000', 'last_peered': '2024-09-11T01:21:37.648831+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:37.648831+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'4900", 'ondisk_log_start': "39'4900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21312454, 'num_objects': 5205, 'num_object_clones': 0, 'num_object_copies': 10410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5205, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11592, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1103}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:43.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:44.682 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:44.682 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39474/168782 objects degraded (23.388%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":505,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84391,"data_bytes":345617161,"bytes_used":1977905152,"bytes_avail":287932387328,"bytes_total":289910292480,"degraded_objects":39474,"degraded_total":168782,"degraded_ratio":0.23387565024706425,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":446},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:44.705 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:44.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:45.267 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:45.270 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:45.302 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-11T01:21:44.581127+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":340288265,"num_objects":83090,"num_object_clones":0,"num_object_copies":166180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83090,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":202155,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10161,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160693,"ondisk_log_size":160693,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1933152,"kb_used_data":869064,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281182368,"statfs":{"total":289910292480,"available":287930744832,"internally_reserved":0,"allocated":889921536,"data_stored":889035840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1,8,9,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24305664,"num_objects":-5934,"num_object_clones":0,"num_object_copies":-11868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5934,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5934,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004745"},"pg_stats":[{"pgid":"2.f","version":"43'15141","reported_seq":15365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.692542+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:42.692542+0000","last_peered":"2024-09-11T01:21:42.692542+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:42.692542+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20877312,"num_objects":5097,"num_object_clones":0,"num_object_copies":10194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15141,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1049}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15235","reported_seq":15316,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.819559+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:40.819559+0000","last_peered":"2024-09-11T01:21:40.819559+0000","last_clean":"2024-09-11T01:21:40.819559+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:40.819559+0000","last_undegraded":"2024-09-11T01:21:40.819559+0000","last_fullsized":"2024-09-11T01:21:40.819559+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21656518,"num_objects":5289,"num_object_clones":0,"num_object_copies":10578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13333,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'14870","reported_seq":15039,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.699553+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:40.699553+0000","last_peered":"2024-09-11T01:21:40.699553+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:40.699553+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'4800","ondisk_log_start":"39'4800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21196824,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12931,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1213}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15117","reported_seq":15163,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.836687+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:40.836687+0000","last_peered":"2024-09-11T01:21:40.836687+0000","last_clean":"2024-09-11T01:21:40.836687+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:40.836687+0000","last_undegraded":"2024-09-11T01:21:40.836687+0000","last_fullsized":"2024-09-11T01:21:40.836687+0000","mapping_epoch":31,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15117,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15141","reported_seq":15263,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:42.652558+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:42.652558+0000","last_peered":"2024-09-11T01:21:42.652558+0000","last_clean":"2024-09-11T01:21:42.652558+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:42.652558+0000","last_undegraded":"2024-09-11T01:21:42.652558+0000","last_fullsized":"2024-09-11T01:21:42.652558+0000","mapping_epoch":31,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21037539,"num_objects":5137,"num_object_clones":0,"num_object_copies":10274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11939,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15086","reported_seq":15351,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.616645+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:42.616645+0000","last_peered":"2024-09-11T01:21:42.616645+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:42.616645+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20976099,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11574,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15604","reported_seq":15888,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.741584+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:42.741584+0000","last_peered":"2024-09-11T01:21:42.741584+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:42.741584+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21332934,"num_objects":5210,"num_object_clones":0,"num_object_copies":10420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5210,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12357,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1004},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15217","reported_seq":15337,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:42.721903+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:42.721903+0000","last_peered":"2024-09-11T01:21:42.721903+0000","last_clean":"2024-09-11T01:21:42.721903+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:42.721903+0000","last_undegraded":"2024-09-11T01:21:42.721903+0000","last_fullsized":"2024-09-11T01:21:42.721903+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12170,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15049","reported_seq":23392,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.465862+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:40.465862+0000","last_peered":"2024-09-11T01:21:40.465862+0000","last_clean":"2024-09-11T01:21:40.465862+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:40.465862+0000","last_undegraded":"2024-09-11T01:21:40.465862+0000","last_fullsized":"2024-09-11T01:21:40.465862+0000","mapping_epoch":40,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21299683,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12916,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15332","reported_seq":15612,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.642858+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:42.642858+0000","last_peered":"2024-09-11T01:21:42.642858+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:42.642858+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21553152,"num_objects":5262,"num_object_clones":0,"num_object_copies":10524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11696,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1232}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'14925","reported_seq":17609,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.790991+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:40.790991+0000","last_peered":"2024-09-11T01:21:40.790991+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:40.790991+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'4900","ondisk_log_start":"39'4900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21357027,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12694,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1294,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15307","reported_seq":15461,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.782184+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:40.782184+0000","last_peered":"2024-09-11T01:21:40.782184+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:40.782184+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21524480,"num_objects":5255,"num_object_clones":0,"num_object_copies":10510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1108}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15176","reported_seq":23647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.459564+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:40.459564+0000","last_peered":"2024-09-11T01:21:40.459564+0000","last_clean":"2024-09-11T01:21:40.459564+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:40.459564+0000","last_undegraded":"2024-09-11T01:21:40.459564+0000","last_fullsized":"2024-09-11T01:21:40.459564+0000","mapping_epoch":42,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21200896,"num_objects":5176,"num_object_clones":0,"num_object_copies":10352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11866,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15445","reported_seq":15730,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.713175+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:42.713175+0000","last_peered":"2024-09-11T01:21:42.713175+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:42.713175+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21303296,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11838,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1084}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15379","reported_seq":15679,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.691546+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:42.691546+0000","last_peered":"2024-09-11T01:21:42.691546+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:42.691546+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21385216,"num_objects":5221,"num_object_clones":0,"num_object_copies":10442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11914,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1156}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15066","reported_seq":15306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.713572+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:42.713572+0000","last_peered":"2024-09-11T01:21:42.713572+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:42.713572+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20775878,"num_objects":5074,"num_object_clones":0,"num_object_copies":10148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11723,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":972},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":340288246,"num_objects":83088,"num_object_clones":0,"num_object_copies":166176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83088,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":202153,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10160,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885796864,"data_stored":885644200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160690,"ondisk_log_size":160690,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603808,"kb_used_data":301384,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93768032,"statfs":{"total":96636764160,"available":96018464768,"internally_reserved":0,"allocated":308617216,"data_stored":308335388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790029,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609204,"kb_used_data":252640,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93762636,"statfs":{"total":96636764160,"available":96012939264,"internally_reserved":0,"allocated":258703360,"data_stored":258381325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":720140,"kb_used_data":315040,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93651700,"statfs":{"total":96636764160,"available":95899340800,"internally_reserved":0,"allocated":322600960,"data_stored":322319127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":414772906},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,1,6,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":321150976,"data_stored":321114846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":256946176,"data_stored":256869844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307699712,"data_stored":307659510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:48.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:48.880 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:48.881 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:48.904 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-11T01:21:48.582594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330879753,"num_objects":80793,"num_object_clones":0,"num_object_copies":161586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80793,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":204452,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10245,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1914952,"kb_used_data":850888,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281200568,"statfs":{"total":289910292480,"available":287949381632,"internally_reserved":0,"allocated":871309312,"data_stored":870423616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,5,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24199168,"num_objects":-5908,"num_object_clones":0,"num_object_copies":-11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5908,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004685"},"pg_stats":[{"pgid":"2.f","version":"43'15263","reported_seq":15487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.738723+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:47.738723+0000","last_peered":"2024-09-11T01:21:47.738723+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:47.738723+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20377600,"num_objects":4975,"num_object_clones":0,"num_object_copies":9950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15383","reported_seq":15464,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.793815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:45.793815+0000","last_peered":"2024-09-11T01:21:45.793815+0000","last_clean":"2024-09-11T01:21:45.793815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:45.793815+0000","last_undegraded":"2024-09-11T01:21:45.793815+0000","last_fullsized":"2024-09-11T01:21:45.793815+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21050310,"num_objects":5141,"num_object_clones":0,"num_object_copies":10282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15018","reported_seq":15187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.770606+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:45.770606+0000","last_peered":"2024-09-11T01:21:45.770606+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:45.770606+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20590616,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13079,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15256","reported_seq":15302,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.806763+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:45.806763+0000","last_peered":"2024-09-11T01:21:45.806763+0000","last_clean":"2024-09-11T01:21:45.806763+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:45.806763+0000","last_undegraded":"2024-09-11T01:21:45.806763+0000","last_fullsized":"2024-09-11T01:21:45.806763+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15289","reported_seq":15411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.697094+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:47.697094+0000","last_peered":"2024-09-11T01:21:47.697094+0000","last_clean":"2024-09-11T01:21:47.697094+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:47.697094+0000","last_undegraded":"2024-09-11T01:21:47.697094+0000","last_fullsized":"2024-09-11T01:21:47.697094+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20431331,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12087,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15239","reported_seq":15506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.768886+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:47.768886+0000","last_peered":"2024-09-11T01:21:47.768886+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:47.768886+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20349411,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11727,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":920},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15749","reported_seq":16037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.731278+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:47.731278+0000","last_peered":"2024-09-11T01:21:47.731278+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:47.731278+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":859},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15340","reported_seq":15460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.748878+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:47.748878+0000","last_peered":"2024-09-11T01:21:47.748878+0000","last_clean":"2024-09-11T01:21:47.748878+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:47.748878+0000","last_undegraded":"2024-09-11T01:21:47.748878+0000","last_fullsized":"2024-09-11T01:21:47.748878+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12293,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15203","reported_seq":23544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.489745+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:45.489745+0000","last_peered":"2024-09-11T01:21:45.489745+0000","last_clean":"2024-09-11T01:21:45.489745+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:45.489745+0000","last_undegraded":"2024-09-11T01:21:45.489745+0000","last_fullsized":"2024-09-11T01:21:45.489745+0000","mapping_epoch":40,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20668899,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13070,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15493","reported_seq":15772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.759120+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:47.759120+0000","last_peered":"2024-09-11T01:21:47.759120+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:47.759120+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20893696,"num_objects":5101,"num_object_clones":0,"num_object_copies":10202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15074","reported_seq":17925,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.802385+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:45.802385+0000","last_peered":"2024-09-11T01:21:45.802385+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:45.802385+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20746723,"num_objects":5066,"num_object_clones":0,"num_object_copies":10132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5066,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12843,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1378,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15438","reported_seq":15592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.829164+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:45.829164+0000","last_peered":"2024-09-11T01:21:45.829164+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:45.829164+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20987904,"num_objects":5124,"num_object_clones":0,"num_object_copies":10248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15305","reported_seq":23776,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.447216+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:45.447216+0000","last_peered":"2024-09-11T01:21:45.447216+0000","last_clean":"2024-09-11T01:21:45.447216+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:45.447216+0000","last_undegraded":"2024-09-11T01:21:45.447216+0000","last_fullsized":"2024-09-11T01:21:45.447216+0000","mapping_epoch":42,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20672512,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11995,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15594","reported_seq":15876,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.752616+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:47.752616+0000","last_peered":"2024-09-11T01:21:47.752616+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:47.752616+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11987,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15533","reported_seq":15833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.722461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:47.722461+0000","last_peered":"2024-09-11T01:21:47.722461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:47.722461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20754432,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12068,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15210","reported_seq":15450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.697821+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:47.697821+0000","last_peered":"2024-09-11T01:21:47.697821+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:47.697821+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20186054,"num_objects":4930,"num_object_clones":0,"num_object_copies":9860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11867,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330879734,"num_objects":80791,"num_object_clones":0,"num_object_copies":161582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80791,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":204450,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10244,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":866955264,"data_stored":866802600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603476,"kb_used_data":296900,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768364,"statfs":{"total":96636764160,"available":96018804736,"internally_reserved":0,"allocated":304025600,"data_stored":303743772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790030,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610548,"kb_used_data":245652,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93761292,"statfs":{"total":96636764160,"available":96011563008,"internally_reserved":0,"allocated":251547648,"data_stored":251225613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700928,"kb_used_data":308336,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93670912,"statfs":{"total":96636764160,"available":95919013888,"internally_reserved":0,"allocated":315736064,"data_stored":315454231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314060800,"data_stored":314024670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249892864,"data_stored":249816532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303001600,"data_stored":302961398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:48.906 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:49.508 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:49.508 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:49.536 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-11T01:21:48.582594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330879753,"num_objects":80793,"num_object_clones":0,"num_object_copies":161586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80793,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":204452,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10245,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1914952,"kb_used_data":850888,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281200568,"statfs":{"total":289910292480,"available":287949381632,"internally_reserved":0,"allocated":871309312,"data_stored":870423616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,5,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24199168,"num_objects":-5908,"num_object_clones":0,"num_object_copies":-11816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5908,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5908,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004685"},"pg_stats":[{"pgid":"2.f","version":"43'15263","reported_seq":15487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.738723+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:47.738723+0000","last_peered":"2024-09-11T01:21:47.738723+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:47.738723+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20377600,"num_objects":4975,"num_object_clones":0,"num_object_copies":9950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15383","reported_seq":15464,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.793815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:45.793815+0000","last_peered":"2024-09-11T01:21:45.793815+0000","last_clean":"2024-09-11T01:21:45.793815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:45.793815+0000","last_undegraded":"2024-09-11T01:21:45.793815+0000","last_fullsized":"2024-09-11T01:21:45.793815+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21050310,"num_objects":5141,"num_object_clones":0,"num_object_copies":10282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15018","reported_seq":15187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.770606+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:45.770606+0000","last_peered":"2024-09-11T01:21:45.770606+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:45.770606+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20590616,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13079,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15256","reported_seq":15302,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.806763+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:45.806763+0000","last_peered":"2024-09-11T01:21:45.806763+0000","last_clean":"2024-09-11T01:21:45.806763+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:45.806763+0000","last_undegraded":"2024-09-11T01:21:45.806763+0000","last_fullsized":"2024-09-11T01:21:45.806763+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15289","reported_seq":15411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.697094+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:47.697094+0000","last_peered":"2024-09-11T01:21:47.697094+0000","last_clean":"2024-09-11T01:21:47.697094+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:47.697094+0000","last_undegraded":"2024-09-11T01:21:47.697094+0000","last_fullsized":"2024-09-11T01:21:47.697094+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20431331,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12087,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15239","reported_seq":15506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.768886+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:47.768886+0000","last_peered":"2024-09-11T01:21:47.768886+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:47.768886+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20349411,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11727,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":920},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15749","reported_seq":16037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.731278+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:47.731278+0000","last_peered":"2024-09-11T01:21:47.731278+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:47.731278+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":859},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15340","reported_seq":15460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.748878+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:47.748878+0000","last_peered":"2024-09-11T01:21:47.748878+0000","last_clean":"2024-09-11T01:21:47.748878+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:47.748878+0000","last_undegraded":"2024-09-11T01:21:47.748878+0000","last_fullsized":"2024-09-11T01:21:47.748878+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12293,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15203","reported_seq":23544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.489745+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:45.489745+0000","last_peered":"2024-09-11T01:21:45.489745+0000","last_clean":"2024-09-11T01:21:45.489745+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:45.489745+0000","last_undegraded":"2024-09-11T01:21:45.489745+0000","last_fullsized":"2024-09-11T01:21:45.489745+0000","mapping_epoch":40,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20668899,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13070,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15493","reported_seq":15772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.759120+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:47.759120+0000","last_peered":"2024-09-11T01:21:47.759120+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:47.759120+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20893696,"num_objects":5101,"num_object_clones":0,"num_object_copies":10202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15074","reported_seq":17925,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.802385+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:45.802385+0000","last_peered":"2024-09-11T01:21:45.802385+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:45.802385+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20746723,"num_objects":5066,"num_object_clones":0,"num_object_copies":10132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5066,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12843,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1378,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15438","reported_seq":15592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.829164+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:45.829164+0000","last_peered":"2024-09-11T01:21:45.829164+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:45.829164+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20987904,"num_objects":5124,"num_object_clones":0,"num_object_copies":10248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15305","reported_seq":23776,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.447216+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:45.447216+0000","last_peered":"2024-09-11T01:21:45.447216+0000","last_clean":"2024-09-11T01:21:45.447216+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:45.447216+0000","last_undegraded":"2024-09-11T01:21:45.447216+0000","last_fullsized":"2024-09-11T01:21:45.447216+0000","mapping_epoch":42,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20672512,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11995,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15594","reported_seq":15876,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.752616+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:47.752616+0000","last_peered":"2024-09-11T01:21:47.752616+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:47.752616+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11987,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15533","reported_seq":15833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.722461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:47.722461+0000","last_peered":"2024-09-11T01:21:47.722461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:47.722461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20754432,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12068,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15210","reported_seq":15450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.697821+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:47.697821+0000","last_peered":"2024-09-11T01:21:47.697821+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:47.697821+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20186054,"num_objects":4930,"num_object_clones":0,"num_object_copies":9860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11867,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330879734,"num_objects":80791,"num_object_clones":0,"num_object_copies":161582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80791,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":204450,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10244,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":866955264,"data_stored":866802600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603476,"kb_used_data":296900,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768364,"statfs":{"total":96636764160,"available":96018804736,"internally_reserved":0,"allocated":304025600,"data_stored":303743772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790030,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610548,"kb_used_data":245652,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93761292,"statfs":{"total":96636764160,"available":96011563008,"internally_reserved":0,"allocated":251547648,"data_stored":251225613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700928,"kb_used_data":308336,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93670912,"statfs":{"total":96636764160,"available":95919013888,"internally_reserved":0,"allocated":315736064,"data_stored":315454231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314060800,"data_stored":314024670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249892864,"data_stored":249816532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303001600,"data_stored":302961398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:49.537 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:49.537 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'15263", 'reported_seq': 15487, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.738723+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:47.738723+0000', 'last_peered': '2024-09-11T01:21:47.738723+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:47.738723+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'5200", 'ondisk_log_start': "39'5200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20377600, 'num_objects': 4975, 'num_object_clones': 0, 'num_object_copies': 9950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15263, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 927}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'15018", 'reported_seq': 15187, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:45.770606+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:45.770606+0000', 'last_peered': '2024-09-11T01:21:45.770606+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:45.770606+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20590616, 'num_objects': 5028, 'num_object_clones': 0, 'num_object_copies': 10056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5028, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13079, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'15239", 'reported_seq': 15506, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.768886+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:47.768886+0000', 'last_peered': '2024-09-11T01:21:47.768886+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:47.768886+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'5200", 'ondisk_log_start': "39'5200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20349411, 'num_objects': 4969, 'num_object_clones': 0, 'num_object_copies': 9938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4969, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11727, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 920}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'15749", 'reported_seq': 16037, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.731278+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:47.731278+0000', 'last_peered': '2024-09-11T01:21:47.731278+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:47.731278+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'5700", 'ondisk_log_start': "39'5700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20739014, 'num_objects': 5065, 'num_object_clones': 0, 'num_object_copies': 10130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5065, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12502, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 859}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'15493", 'reported_seq': 15772, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.759120+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:47.759120+0000', 'last_peered': '2024-09-11T01:21:47.759120+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:47.759120+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'5400", 'ondisk_log_start': "39'5400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20893696, 'num_objects': 5101, 'num_object_clones': 0, 'num_object_copies': 10202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11857, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1071}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'15074", 'reported_seq': 17925, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:45.802385+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:45.802385+0000', 'last_peered': '2024-09-11T01:21:45.802385+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:45.802385+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'5000", 'ondisk_log_start': "39'5000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20746723, 'num_objects': 5066, 'num_object_clones': 0, 'num_object_copies': 10132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2650, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5066, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12843, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1378, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2416}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:49.538 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15438", 'reported_seq': 15592, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:45.829164+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:45.829164+0000', 'last_peered': '2024-09-11T01:21:45.829164+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:45.829164+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5400", 'ondisk_log_start': "39'5400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20987904, 'num_objects': 5124, 'num_object_clones': 0, 'num_object_copies': 10248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13075, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 977}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'15594", 'reported_seq': 15876, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.752616+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:47.752616+0000', 'last_peered': '2024-09-11T01:21:47.752616+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:47.752616+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'5500", 'ondisk_log_start': "39'5500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20692992, 'num_objects': 5052, 'num_object_clones': 0, 'num_object_copies': 10104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11987, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'15533", 'reported_seq': 15833, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.722461+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:47.722461+0000', 'last_peered': '2024-09-11T01:21:47.722461+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:47.722461+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'5500", 'ondisk_log_start': "39'5500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20754432, 'num_objects': 5067, 'num_object_clones': 0, 'num_object_copies': 10134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12068, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:49.539 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'15210", 'reported_seq': 15450, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.697821+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:47.697821+0000', 'last_peered': '2024-09-11T01:21:47.697821+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:47.697821+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'5200", 'ondisk_log_start': "39'5200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20186054, 'num_objects': 4930, 'num_object_clones': 0, 'num_object_copies': 9860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4930, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11867, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 828}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:49.539 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:50.241 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:50.241 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39390/161586 objects degraded (24.377%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":510,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80793,"data_bytes":330879753,"bytes_used":1960910848,"bytes_avail":287949381632,"bytes_total":289910292480,"degraded_objects":39390,"degraded_total":161586,"degraded_ratio":0.24377111878504326,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":492},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:50.266 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:50.267 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:50.836 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:50.839 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:50.863 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-11T01:21:50.583391+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330879753,"num_objects":80793,"num_object_clones":0,"num_object_copies":161586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80793,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":204452,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10245,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1914952,"kb_used_data":850888,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281200568,"statfs":{"total":289910292480,"available":287949381632,"internally_reserved":0,"allocated":871309312,"data_stored":870423616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,5,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18853888,"num_objects":-4603,"num_object_clones":0,"num_object_copies":-9206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-177,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4603,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004533"},"pg_stats":[{"pgid":"2.f","version":"43'15263","reported_seq":15487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.738723+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:47.738723+0000","last_peered":"2024-09-11T01:21:47.738723+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:47.738723+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20377600,"num_objects":4975,"num_object_clones":0,"num_object_copies":9950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15383","reported_seq":15464,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.793815+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:45.793815+0000","last_peered":"2024-09-11T01:21:45.793815+0000","last_clean":"2024-09-11T01:21:45.793815+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:45.793815+0000","last_undegraded":"2024-09-11T01:21:45.793815+0000","last_fullsized":"2024-09-11T01:21:45.793815+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21050310,"num_objects":5141,"num_object_clones":0,"num_object_copies":10282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15018","reported_seq":15187,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.770606+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:45.770606+0000","last_peered":"2024-09-11T01:21:45.770606+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:45.770606+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20590616,"num_objects":5028,"num_object_clones":0,"num_object_copies":10056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13079,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1065}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15256","reported_seq":15302,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.806763+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:45.806763+0000","last_peered":"2024-09-11T01:21:45.806763+0000","last_clean":"2024-09-11T01:21:45.806763+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:45.806763+0000","last_undegraded":"2024-09-11T01:21:45.806763+0000","last_fullsized":"2024-09-11T01:21:45.806763+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15289","reported_seq":15411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.697094+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:47.697094+0000","last_peered":"2024-09-11T01:21:47.697094+0000","last_clean":"2024-09-11T01:21:47.697094+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:47.697094+0000","last_undegraded":"2024-09-11T01:21:47.697094+0000","last_fullsized":"2024-09-11T01:21:47.697094+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20431331,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12087,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15239","reported_seq":15506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.768886+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:47.768886+0000","last_peered":"2024-09-11T01:21:47.768886+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:47.768886+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20349411,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11727,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":920},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15749","reported_seq":16037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.731278+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:47.731278+0000","last_peered":"2024-09-11T01:21:47.731278+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:47.731278+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":859},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15340","reported_seq":15460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.748878+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:47.748878+0000","last_peered":"2024-09-11T01:21:47.748878+0000","last_clean":"2024-09-11T01:21:47.748878+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:47.748878+0000","last_undegraded":"2024-09-11T01:21:47.748878+0000","last_fullsized":"2024-09-11T01:21:47.748878+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12293,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15203","reported_seq":23544,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.489745+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:45.489745+0000","last_peered":"2024-09-11T01:21:45.489745+0000","last_clean":"2024-09-11T01:21:45.489745+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:45.489745+0000","last_undegraded":"2024-09-11T01:21:45.489745+0000","last_fullsized":"2024-09-11T01:21:45.489745+0000","mapping_epoch":40,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20668899,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13070,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15493","reported_seq":15772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.759120+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:47.759120+0000","last_peered":"2024-09-11T01:21:47.759120+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:47.759120+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20893696,"num_objects":5101,"num_object_clones":0,"num_object_copies":10202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15074","reported_seq":17925,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.802385+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:45.802385+0000","last_peered":"2024-09-11T01:21:45.802385+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:45.802385+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5000","ondisk_log_start":"39'5000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20746723,"num_objects":5066,"num_object_clones":0,"num_object_copies":10132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5066,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12843,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1378,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15438","reported_seq":15592,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.829164+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:45.829164+0000","last_peered":"2024-09-11T01:21:45.829164+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:45.829164+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20987904,"num_objects":5124,"num_object_clones":0,"num_object_copies":10248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13075,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15305","reported_seq":23776,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.447216+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:45.447216+0000","last_peered":"2024-09-11T01:21:45.447216+0000","last_clean":"2024-09-11T01:21:45.447216+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:45.447216+0000","last_undegraded":"2024-09-11T01:21:45.447216+0000","last_fullsized":"2024-09-11T01:21:45.447216+0000","mapping_epoch":42,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20672512,"num_objects":5047,"num_object_clones":0,"num_object_copies":10094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11995,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15594","reported_seq":15876,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.752616+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:47.752616+0000","last_peered":"2024-09-11T01:21:47.752616+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:47.752616+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11987,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15533","reported_seq":15833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.722461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:47.722461+0000","last_peered":"2024-09-11T01:21:47.722461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:47.722461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20754432,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12068,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680422+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680422+0000","last_peered":"2024-09-11T01:20:45.680422+0000","last_clean":"2024-09-11T01:20:45.680422+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680422+0000","last_undegraded":"2024-09-11T01:20:45.680422+0000","last_fullsized":"2024-09-11T01:20:45.680422+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15210","reported_seq":15450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.697821+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:47.697821+0000","last_peered":"2024-09-11T01:21:47.697821+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:47.697821+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20186054,"num_objects":4930,"num_object_clones":0,"num_object_copies":9860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11867,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330879734,"num_objects":80791,"num_object_clones":0,"num_object_copies":161582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80791,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":204450,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10244,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":866955264,"data_stored":866802600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603476,"kb_used_data":296900,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93768364,"statfs":{"total":96636764160,"available":96018804736,"internally_reserved":0,"allocated":304025600,"data_stored":303743772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790030,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610548,"kb_used_data":245652,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93761292,"statfs":{"total":96636764160,"available":96011563008,"internally_reserved":0,"allocated":251547648,"data_stored":251225613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700928,"kb_used_data":308336,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93670912,"statfs":{"total":96636764160,"available":95919013888,"internally_reserved":0,"allocated":315736064,"data_stored":315454231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314060800,"data_stored":314024670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":249892864,"data_stored":249816532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303001600,"data_stored":302961398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:53.865 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:54.440 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:54.443 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:54.462 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-11T01:21:52.584041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":326857481,"num_objects":79811,"num_object_clones":0,"num_object_copies":159622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79811,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":205434,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10332,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1895044,"kb_used_data":839304,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281220476,"statfs":{"total":289910292480,"available":287969767424,"internally_reserved":0,"allocated":859447296,"data_stored":858561600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1081003015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,4,9,9,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21659648,"num_objects":-5288,"num_object_clones":0,"num_object_copies":-10576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5288,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004534"},"pg_stats":[{"pgid":"2.f","version":"43'15263","reported_seq":15487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.738723+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:47.738723+0000","last_peered":"2024-09-11T01:21:47.738723+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:47.738723+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20377600,"num_objects":4975,"num_object_clones":0,"num_object_copies":9950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15527","reported_seq":15607,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.835524+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:50.835524+0000","last_peered":"2024-09-11T01:21:50.835524+0000","last_clean":"2024-09-11T01:21:50.835524+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:50.835524+0000","last_undegraded":"2024-09-11T01:21:50.835524+0000","last_fullsized":"2024-09-11T01:21:50.835524+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20460486,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13625,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15165","reported_seq":15334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.835592+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:50.835592+0000","last_peered":"2024-09-11T01:21:50.835592+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:50.835592+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988504,"num_objects":4881,"num_object_clones":0,"num_object_copies":9762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13226,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":918}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15394","reported_seq":15439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.745268+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:50.745268+0000","last_peered":"2024-09-11T01:21:50.745268+0000","last_clean":"2024-09-11T01:21:50.745268+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:50.745268+0000","last_undegraded":"2024-09-11T01:21:50.745268+0000","last_fullsized":"2024-09-11T01:21:50.745268+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20348928,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15289","reported_seq":15411,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.697094+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:47.697094+0000","last_peered":"2024-09-11T01:21:47.697094+0000","last_clean":"2024-09-11T01:21:47.697094+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:47.697094+0000","last_undegraded":"2024-09-11T01:21:47.697094+0000","last_fullsized":"2024-09-11T01:21:47.697094+0000","mapping_epoch":31,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20431331,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12087,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15239","reported_seq":15506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.768886+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:47.768886+0000","last_peered":"2024-09-11T01:21:47.768886+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:47.768886+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20349411,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11727,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":920},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15749","reported_seq":16037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.731278+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:47.731278+0000","last_peered":"2024-09-11T01:21:47.731278+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:47.731278+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20739014,"num_objects":5065,"num_object_clones":0,"num_object_copies":10130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5065,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12502,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":859},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15340","reported_seq":15460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.748878+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:47.748878+0000","last_peered":"2024-09-11T01:21:47.748878+0000","last_clean":"2024-09-11T01:21:47.748878+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:47.748878+0000","last_undegraded":"2024-09-11T01:21:47.748878+0000","last_fullsized":"2024-09-11T01:21:47.748878+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20824064,"num_objects":5084,"num_object_clones":0,"num_object_copies":10168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12293,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15338","reported_seq":23682,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.547952+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:50.547952+0000","last_peered":"2024-09-11T01:21:50.547952+0000","last_clean":"2024-09-11T01:21:50.547952+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:50.547952+0000","last_undegraded":"2024-09-11T01:21:50.547952+0000","last_fullsized":"2024-09-11T01:21:50.547952+0000","mapping_epoch":40,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115939,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13205,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15493","reported_seq":15772,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.759120+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:47.759120+0000","last_peered":"2024-09-11T01:21:47.759120+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:47.759120+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20893696,"num_objects":5101,"num_object_clones":0,"num_object_copies":10202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15211","reported_seq":18237,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.806380+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:50.806380+0000","last_peered":"2024-09-11T01:21:50.806380+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:50.806380+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185571,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12980,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1465,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15573","reported_seq":15727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.839712+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:50.839712+0000","last_peered":"2024-09-11T01:21:50.839712+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:50.839712+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20434944,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":842}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15451","reported_seq":23922,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.516016+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:50.516016+0000","last_peered":"2024-09-11T01:21:50.516016+0000","last_clean":"2024-09-11T01:21:50.516016+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:50.516016+0000","last_undegraded":"2024-09-11T01:21:50.516016+0000","last_fullsized":"2024-09-11T01:21:50.516016+0000","mapping_epoch":42,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15594","reported_seq":15876,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.752616+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:47.752616+0000","last_peered":"2024-09-11T01:21:47.752616+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:47.752616+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11987,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15533","reported_seq":15833,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.722461+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:47.722461+0000","last_peered":"2024-09-11T01:21:47.722461+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:47.722461+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20754432,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12068,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15210","reported_seq":15450,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.697821+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:47.697821+0000","last_peered":"2024-09-11T01:21:47.697821+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:47.697821+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20186054,"num_objects":4930,"num_object_clones":0,"num_object_copies":9860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4930,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11867,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":326857462,"num_objects":79809,"num_object_clones":0,"num_object_copies":159618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79809,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":205432,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10331,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":855212032,"data_stored":855059368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603104,"kb_used_data":292368,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93768736,"statfs":{"total":96636764160,"available":96019185664,"internally_reserved":0,"allocated":299384832,"data_stored":299103004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591012,"kb_used_data":238600,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93780828,"statfs":{"total":96636764160,"available":96031567872,"internally_reserved":0,"allocated":244326400,"data_stored":244004365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700928,"kb_used_data":308336,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93670912,"statfs":{"total":96636764160,"available":95919013888,"internally_reserved":0,"allocated":315736064,"data_stored":315454231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":314060800,"data_stored":314024670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":242692096,"data_stored":242615764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298459136,"data_stored":298418934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:54.463 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:55.046 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:55.046 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:55.067 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T01:21:54.584755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":321442569,"num_objects":78489,"num_object_clones":0,"num_object_copies":156978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78489,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":206756,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10332,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1892080,"kb_used_data":832180,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281223440,"statfs":{"total":289910292480,"available":287972802560,"internally_reserved":0,"allocated":852152320,"data_stored":851266624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,7,9,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24174592,"num_objects":-5902,"num_object_clones":0,"num_object_copies":-11804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004557"},"pg_stats":[{"pgid":"2.f","version":"43'15407","reported_seq":15631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.798506+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:52.798506+0000","last_peered":"2024-09-11T01:21:52.798506+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:52.798506+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15407,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15527","reported_seq":15607,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.835524+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:50.835524+0000","last_peered":"2024-09-11T01:21:50.835524+0000","last_clean":"2024-09-11T01:21:50.835524+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:50.835524+0000","last_undegraded":"2024-09-11T01:21:50.835524+0000","last_fullsized":"2024-09-11T01:21:50.835524+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20460486,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13625,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15165","reported_seq":15334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.835592+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:50.835592+0000","last_peered":"2024-09-11T01:21:50.835592+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:50.835592+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988504,"num_objects":4881,"num_object_clones":0,"num_object_copies":9762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13226,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":918}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15394","reported_seq":15439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.745268+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:50.745268+0000","last_peered":"2024-09-11T01:21:50.745268+0000","last_clean":"2024-09-11T01:21:50.745268+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:50.745268+0000","last_undegraded":"2024-09-11T01:21:50.745268+0000","last_fullsized":"2024-09-11T01:21:50.745268+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20348928,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15433","reported_seq":15555,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.798317+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:52.798317+0000","last_peered":"2024-09-11T01:21:52.798317+0000","last_clean":"2024-09-11T01:21:52.798317+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:52.798317+0000","last_undegraded":"2024-09-11T01:21:52.798317+0000","last_fullsized":"2024-09-11T01:21:52.798317+0000","mapping_epoch":31,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19841507,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12231,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15381","reported_seq":15648,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.821894+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:52.821894+0000","last_peered":"2024-09-11T01:21:52.821894+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:52.821894+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19767779,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11869,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15898","reported_seq":16186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.725624+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:52.725624+0000","last_peered":"2024-09-11T01:21:52.725624+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:52.725624+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20128710,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12651,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":710},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15485","reported_seq":15605,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.831776+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:52.831776+0000","last_peered":"2024-09-11T01:21:52.831776+0000","last_clean":"2024-09-11T01:21:52.831776+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:52.831776+0000","last_undegraded":"2024-09-11T01:21:52.831776+0000","last_fullsized":"2024-09-11T01:21:52.831776+0000","mapping_epoch":31,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20230144,"num_objects":4939,"num_object_clones":0,"num_object_copies":9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12438,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15338","reported_seq":23682,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.547952+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:50.547952+0000","last_peered":"2024-09-11T01:21:50.547952+0000","last_clean":"2024-09-11T01:21:50.547952+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:50.547952+0000","last_undegraded":"2024-09-11T01:21:50.547952+0000","last_fullsized":"2024-09-11T01:21:50.547952+0000","mapping_epoch":40,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115939,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13205,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15648","reported_seq":15925,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.814806+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:52.814806+0000","last_peered":"2024-09-11T01:21:52.814806+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:52.814806+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12012,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15211","reported_seq":18237,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.806380+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:50.806380+0000","last_peered":"2024-09-11T01:21:50.806380+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:50.806380+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185571,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12980,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1465,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15573","reported_seq":15727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.839712+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:50.839712+0000","last_peered":"2024-09-11T01:21:50.839712+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:50.839712+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20434944,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":842}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15451","reported_seq":23922,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.516016+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:50.516016+0000","last_peered":"2024-09-11T01:21:50.516016+0000","last_clean":"2024-09-11T01:21:50.516016+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:50.516016+0000","last_undegraded":"2024-09-11T01:21:50.516016+0000","last_fullsized":"2024-09-11T01:21:50.516016+0000","mapping_epoch":42,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15745","reported_seq":16030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.762532+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:52.762532+0000","last_peered":"2024-09-11T01:21:52.762532+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:52.762532+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12138,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":784}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15684","reported_seq":15984,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.788834+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:52.788834+0000","last_peered":"2024-09-11T01:21:52.788834+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:52.788834+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20135936,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12219,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15351","reported_seq":15591,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.798166+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:52.798166+0000","last_peered":"2024-09-11T01:21:52.798166+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:52.798166+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19608518,"num_objects":4789,"num_object_clones":0,"num_object_copies":9578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12008,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":687},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":321442550,"num_objects":78487,"num_object_clones":0,"num_object_copies":156974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78487,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":206754,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10331,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":847958016,"data_stored":847805352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603104,"kb_used_data":292368,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93768736,"statfs":{"total":96636764160,"available":96019185664,"internally_reserved":0,"allocated":299384832,"data_stored":299103004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591012,"kb_used_data":238600,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93780828,"statfs":{"total":96636764160,"available":96031567872,"internally_reserved":0,"allocated":244326400,"data_stored":244004365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697964,"kb_used_data":301212,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93673876,"statfs":{"total":96636764160,"available":95922049024,"internally_reserved":0,"allocated":308441088,"data_stored":308159255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306806784,"data_stored":306770654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":242692096,"data_stored":242615764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298459136,"data_stored":298418934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:55.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:55.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'15407", 'reported_seq': 15631, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.798506+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:52.798506+0000', 'last_peered': '2024-09-11T01:21:52.798506+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:52.798506+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'5400", 'ondisk_log_start': "39'5400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19787776, 'num_objects': 4831, 'num_object_clones': 0, 'num_object_copies': 9662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4831, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15407, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 783}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:55.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'15165", 'reported_seq': 15334, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:50.835592+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:50.835592+0000', 'last_peered': '2024-09-11T01:21:50.835592+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:50.835592+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'5100", 'ondisk_log_start': "39'5100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19988504, 'num_objects': 4881, 'num_object_clones': 0, 'num_object_copies': 9762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4881, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13226, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 918}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:55.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:55.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'15381", 'reported_seq': 15648, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.821894+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:52.821894+0000', 'last_peered': '2024-09-11T01:21:52.821894+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:52.821894+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'5300", 'ondisk_log_start': "39'5300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19767779, 'num_objects': 4827, 'num_object_clones': 0, 'num_object_copies': 9654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4827, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11869, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 778}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'15898", 'reported_seq': 16186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.725624+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:52.725624+0000', 'last_peered': '2024-09-11T01:21:52.725624+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:52.725624+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'5800", 'ondisk_log_start': "39'5800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20128710, 'num_objects': 4916, 'num_object_clones': 0, 'num_object_copies': 9832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4916, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12651, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 710}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'15648", 'reported_seq': 15925, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.814806+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:52.814806+0000', 'last_peered': '2024-09-11T01:21:52.814806+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:52.814806+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20258816, 'num_objects': 4946, 'num_object_clones': 0, 'num_object_copies': 9892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4946, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12012, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 916}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'15211", 'reported_seq': 18237, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:50.806380+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:50.806380+0000', 'last_peered': '2024-09-11T01:21:50.806380+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:50.806380+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'5200", 'ondisk_log_start': "39'5200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20185571, 'num_objects': 4929, 'num_object_clones': 0, 'num_object_copies': 9858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2563, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4929, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12980, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1465, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2366}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15573", 'reported_seq': 15727, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:50.839712+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:50.839712+0000', 'last_peered': '2024-09-11T01:21:50.839712+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:50.839712+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5500", 'ondisk_log_start': "39'5500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20434944, 'num_objects': 4989, 'num_object_clones': 0, 'num_object_copies': 9978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13210, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 842}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'15745", 'reported_seq': 16030, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.762532+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:52.762532+0000', 'last_peered': '2024-09-11T01:21:52.762532+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:52.762532+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'5700", 'ondisk_log_start': "39'5700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20074496, 'num_objects': 4901, 'num_object_clones': 0, 'num_object_copies': 9802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12138, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 784}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:55.070 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'15684", 'reported_seq': 15984, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.788834+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:52.788834+0000', 'last_peered': '2024-09-11T01:21:52.788834+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:52.788834+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20135936, 'num_objects': 4916, 'num_object_clones': 0, 'num_object_copies': 9832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12219, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 851}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.070 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:55.070 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'15351", 'reported_seq': 15591, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:52.798166+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:52.798166+0000', 'last_peered': '2024-09-11T01:21:52.798166+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:52.798166+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'5300", 'ondisk_log_start': "39'5300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19608518, 'num_objects': 4789, 'num_object_clones': 0, 'num_object_copies': 9578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4789, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12008, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 687}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.070 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:55.815 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:55.815 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39303/156978 objects degraded (25.037%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78489,"data_bytes":321442569,"bytes_used":1937489920,"bytes_avail":287972802560,"bytes_total":289910292480,"degraded_objects":39303,"degraded_total":156978,"degraded_ratio":0.25037266368535721,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":491},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:55.843 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:55.843 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:56.430 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:21:56.431 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:21:56.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T01:21:54.584755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":321442569,"num_objects":78489,"num_object_clones":0,"num_object_copies":156978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78489,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":206756,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10332,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1892080,"kb_used_data":832180,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281223440,"statfs":{"total":289910292480,"available":287972802560,"internally_reserved":0,"allocated":852152320,"data_stored":851266624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,7,9,7,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24174592,"num_objects":-5902,"num_object_clones":0,"num_object_copies":-11804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004557"},"pg_stats":[{"pgid":"2.f","version":"43'15407","reported_seq":15631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.798506+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:52.798506+0000","last_peered":"2024-09-11T01:21:52.798506+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:52.798506+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19787776,"num_objects":4831,"num_object_clones":0,"num_object_copies":9662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15407,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15527","reported_seq":15607,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.835524+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:50.835524+0000","last_peered":"2024-09-11T01:21:50.835524+0000","last_clean":"2024-09-11T01:21:50.835524+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:50.835524+0000","last_undegraded":"2024-09-11T01:21:50.835524+0000","last_fullsized":"2024-09-11T01:21:50.835524+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20460486,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13625,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15165","reported_seq":15334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.835592+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:50.835592+0000","last_peered":"2024-09-11T01:21:50.835592+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:50.835592+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5100","ondisk_log_start":"39'5100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19988504,"num_objects":4881,"num_object_clones":0,"num_object_copies":9762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13226,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":918}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15394","reported_seq":15439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.745268+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:50.745268+0000","last_peered":"2024-09-11T01:21:50.745268+0000","last_clean":"2024-09-11T01:21:50.745268+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:50.745268+0000","last_undegraded":"2024-09-11T01:21:50.745268+0000","last_fullsized":"2024-09-11T01:21:50.745268+0000","mapping_epoch":31,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20348928,"num_objects":4968,"num_object_clones":0,"num_object_copies":9936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15394,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15433","reported_seq":15555,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.798317+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:52.798317+0000","last_peered":"2024-09-11T01:21:52.798317+0000","last_clean":"2024-09-11T01:21:52.798317+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:52.798317+0000","last_undegraded":"2024-09-11T01:21:52.798317+0000","last_fullsized":"2024-09-11T01:21:52.798317+0000","mapping_epoch":31,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19841507,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12231,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15381","reported_seq":15648,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.821894+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:52.821894+0000","last_peered":"2024-09-11T01:21:52.821894+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:52.821894+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19767779,"num_objects":4827,"num_object_clones":0,"num_object_copies":9654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4827,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11869,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":778},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'15898","reported_seq":16186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.725624+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:52.725624+0000","last_peered":"2024-09-11T01:21:52.725624+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:52.725624+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20128710,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12651,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":710},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15485","reported_seq":15605,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.831776+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:52.831776+0000","last_peered":"2024-09-11T01:21:52.831776+0000","last_clean":"2024-09-11T01:21:52.831776+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:52.831776+0000","last_undegraded":"2024-09-11T01:21:52.831776+0000","last_fullsized":"2024-09-11T01:21:52.831776+0000","mapping_epoch":31,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20230144,"num_objects":4939,"num_object_clones":0,"num_object_copies":9878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12438,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15338","reported_seq":23682,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.547952+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:50.547952+0000","last_peered":"2024-09-11T01:21:50.547952+0000","last_clean":"2024-09-11T01:21:50.547952+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:50.547952+0000","last_undegraded":"2024-09-11T01:21:50.547952+0000","last_fullsized":"2024-09-11T01:21:50.547952+0000","mapping_epoch":40,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20115939,"num_objects":4912,"num_object_clones":0,"num_object_copies":9824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13205,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15648","reported_seq":15925,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.814806+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:52.814806+0000","last_peered":"2024-09-11T01:21:52.814806+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:52.814806+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20258816,"num_objects":4946,"num_object_clones":0,"num_object_copies":9892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12012,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":916}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15211","reported_seq":18237,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.806380+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:50.806380+0000","last_peered":"2024-09-11T01:21:50.806380+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:50.806380+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5200","ondisk_log_start":"39'5200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185571,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2563,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12980,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1465,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15573","reported_seq":15727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.839712+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:50.839712+0000","last_peered":"2024-09-11T01:21:50.839712+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:50.839712+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20434944,"num_objects":4989,"num_object_clones":0,"num_object_copies":9978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13210,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":842}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15451","reported_seq":23922,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.516016+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:50.516016+0000","last_peered":"2024-09-11T01:21:50.516016+0000","last_clean":"2024-09-11T01:21:50.516016+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:50.516016+0000","last_undegraded":"2024-09-11T01:21:50.516016+0000","last_fullsized":"2024-09-11T01:21:50.516016+0000","mapping_epoch":42,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15745","reported_seq":16030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.762532+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:52.762532+0000","last_peered":"2024-09-11T01:21:52.762532+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:52.762532+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20074496,"num_objects":4901,"num_object_clones":0,"num_object_copies":9802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12138,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":784}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15684","reported_seq":15984,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.788834+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:52.788834+0000","last_peered":"2024-09-11T01:21:52.788834+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:52.788834+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20135936,"num_objects":4916,"num_object_clones":0,"num_object_copies":9832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12219,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15351","reported_seq":15591,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:52.798166+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:52.798166+0000","last_peered":"2024-09-11T01:21:52.798166+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:52.798166+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19608518,"num_objects":4789,"num_object_clones":0,"num_object_copies":9578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12008,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":687},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":321442550,"num_objects":78487,"num_object_clones":0,"num_object_copies":156974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78487,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":206754,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10331,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":847958016,"data_stored":847805352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603104,"kb_used_data":292368,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93768736,"statfs":{"total":96636764160,"available":96019185664,"internally_reserved":0,"allocated":299384832,"data_stored":299103004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591012,"kb_used_data":238600,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93780828,"statfs":{"total":96636764160,"available":96031567872,"internally_reserved":0,"allocated":244326400,"data_stored":244004365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697964,"kb_used_data":301212,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93673876,"statfs":{"total":96636764160,"available":95922049024,"internally_reserved":0,"allocated":308441088,"data_stored":308159255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,2,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":306806784,"data_stored":306770654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":242692096,"data_stored":242615764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298459136,"data_stored":298418934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:59.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:00.050 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:00.052 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:00.076 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-11T01:21:58.586471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":312025865,"num_objects":76190,"num_object_clones":0,"num_object_copies":152380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76190,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":209055,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10429,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160593,"ondisk_log_size":160593,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1877636,"kb_used_data":813572,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281237884,"statfs":{"total":289910292480,"available":287987593216,"internally_reserved":0,"allocated":833097728,"data_stored":832212032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,5,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24174592,"num_objects":-5902,"num_object_clones":0,"num_object_copies":-11804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5902,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004585"},"pg_stats":[{"pgid":"2.f","version":"43'15543","reported_seq":15767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.796620+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:57.796620+0000","last_peered":"2024-09-11T01:21:57.796620+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:57.796620+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19230720,"num_objects":4695,"num_object_clones":0,"num_object_copies":9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15543,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15680","reported_seq":15761,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.856542+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:55.856542+0000","last_peered":"2024-09-11T01:21:55.856542+0000","last_clean":"2024-09-11T01:21:55.856542+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:55.856542+0000","last_undegraded":"2024-09-11T01:21:55.856542+0000","last_fullsized":"2024-09-11T01:21:55.856542+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19833798,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13778,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15301","reported_seq":15470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.767539+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:55.767539+0000","last_peered":"2024-09-11T01:21:55.767539+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:55.767539+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431448,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13362,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15540","reported_seq":15586,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.827697+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:55.827697+0000","last_peered":"2024-09-11T01:21:55.827697+0000","last_clean":"2024-09-11T01:21:55.827697+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:55.827697+0000","last_undegraded":"2024-09-11T01:21:55.827697+0000","last_fullsized":"2024-09-11T01:21:55.827697+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15567","reported_seq":15689,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.834551+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:57.834551+0000","last_peered":"2024-09-11T01:21:57.834551+0000","last_clean":"2024-09-11T01:21:57.834551+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:57.834551+0000","last_undegraded":"2024-09-11T01:21:57.834551+0000","last_fullsized":"2024-09-11T01:21:57.834551+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292643,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12365,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15515","reported_seq":15782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.829664+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:57.829664+0000","last_peered":"2024-09-11T01:21:57.829664+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:57.829664+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218915,"num_objects":4693,"num_object_clones":0,"num_object_copies":9386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12003,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":644},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16032","reported_seq":16320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.807971+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:57.807971+0000","last_peered":"2024-09-11T01:21:57.807971+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:57.807971+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579846,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12785,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15635","reported_seq":15755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.754788+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:57.754788+0000","last_peered":"2024-09-11T01:21:57.754788+0000","last_clean":"2024-09-11T01:21:57.754788+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:57.754788+0000","last_undegraded":"2024-09-11T01:21:57.754788+0000","last_fullsized":"2024-09-11T01:21:57.754788+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19615744,"num_objects":4789,"num_object_clones":0,"num_object_copies":9578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15470","reported_seq":23814,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.589471+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:55.589471+0000","last_peered":"2024-09-11T01:21:55.589471+0000","last_clean":"2024-09-11T01:21:55.589471+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:55.589471+0000","last_undegraded":"2024-09-11T01:21:55.589471+0000","last_fullsized":"2024-09-11T01:21:55.589471+0000","mapping_epoch":40,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19575267,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13337,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15813","reported_seq":16092,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.872337+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:57.872337+0000","last_peered":"2024-09-11T01:21:57.872337+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:57.872337+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12177,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15359","reported_seq":18579,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.821740+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:55.821740+0000","last_peered":"2024-09-11T01:21:55.821740+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:55.821740+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579363,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13128,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15706","reported_seq":15860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.811576+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:55.811576+0000","last_peered":"2024-09-11T01:21:55.811576+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:55.811576+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19890176,"num_objects":4856,"num_object_clones":0,"num_object_copies":9712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15607","reported_seq":24076,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.615271+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:55.615271+0000","last_peered":"2024-09-11T01:21:55.615271+0000","last_clean":"2024-09-11T01:21:55.615271+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:55.615271+0000","last_undegraded":"2024-09-11T01:21:55.615271+0000","last_fullsized":"2024-09-11T01:21:55.615271+0000","mapping_epoch":42,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12297,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15901","reported_seq":16186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.808201+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:57.808201+0000","last_peered":"2024-09-11T01:21:57.808201+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:57.808201+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12294,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15827","reported_seq":16126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.867775+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:57.867775+0000","last_peered":"2024-09-11T01:21:57.867775+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:57.867775+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12362,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15494","reported_seq":15734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.788766+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:57.788766+0000","last_peered":"2024-09-11T01:21:57.788766+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:57.788766+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19022790,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12151,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":544},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":312025846,"num_objects":76188,"num_object_clones":0,"num_object_copies":152376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76188,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":209053,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10428,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829022208,"data_stored":828869544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160590,"ondisk_log_size":160590,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586052,"kb_used_data":287792,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93785788,"statfs":{"total":96636764160,"available":96036646912,"internally_reserved":0,"allocated":294699008,"data_stored":294417180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790032,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592156,"kb_used_data":231436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93779684,"statfs":{"total":96636764160,"available":96030396416,"internally_reserved":0,"allocated":236990464,"data_stored":236668429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699428,"kb_used_data":294344,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93672412,"statfs":{"total":96636764160,"available":95920549888,"internally_reserved":0,"allocated":301408256,"data_stored":301126423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":414772906},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299728896,"data_stored":299692766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":235532288,"data_stored":235455956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293761024,"data_stored":293720822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:00.077 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:00.670 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:00.673 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:00.699 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-11T01:22:00.587414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":312025865,"num_objects":76190,"num_object_clones":0,"num_object_copies":152380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76190,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":209055,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10429,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160593,"ondisk_log_size":160593,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1877636,"kb_used_data":813572,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281237884,"statfs":{"total":289910292480,"available":287987593216,"internally_reserved":0,"allocated":833097728,"data_stored":832212032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,5,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18853888,"num_objects":-4603,"num_object_clones":0,"num_object_copies":-9206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4603,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004820"},"pg_stats":[{"pgid":"2.f","version":"43'15543","reported_seq":15767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.796620+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:57.796620+0000","last_peered":"2024-09-11T01:21:57.796620+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:57.796620+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19230720,"num_objects":4695,"num_object_clones":0,"num_object_copies":9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15543,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15680","reported_seq":15761,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.856542+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:55.856542+0000","last_peered":"2024-09-11T01:21:55.856542+0000","last_clean":"2024-09-11T01:21:55.856542+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:55.856542+0000","last_undegraded":"2024-09-11T01:21:55.856542+0000","last_fullsized":"2024-09-11T01:21:55.856542+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19833798,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13778,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15301","reported_seq":15470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.767539+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:55.767539+0000","last_peered":"2024-09-11T01:21:55.767539+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:55.767539+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431448,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13362,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15540","reported_seq":15586,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.827697+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:55.827697+0000","last_peered":"2024-09-11T01:21:55.827697+0000","last_clean":"2024-09-11T01:21:55.827697+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:55.827697+0000","last_undegraded":"2024-09-11T01:21:55.827697+0000","last_fullsized":"2024-09-11T01:21:55.827697+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15567","reported_seq":15689,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.834551+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:57.834551+0000","last_peered":"2024-09-11T01:21:57.834551+0000","last_clean":"2024-09-11T01:21:57.834551+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:57.834551+0000","last_undegraded":"2024-09-11T01:21:57.834551+0000","last_fullsized":"2024-09-11T01:21:57.834551+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292643,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12365,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15515","reported_seq":15782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.829664+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:57.829664+0000","last_peered":"2024-09-11T01:21:57.829664+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:57.829664+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218915,"num_objects":4693,"num_object_clones":0,"num_object_copies":9386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12003,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":644},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16032","reported_seq":16320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.807971+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:57.807971+0000","last_peered":"2024-09-11T01:21:57.807971+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:57.807971+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579846,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12785,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15635","reported_seq":15755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.754788+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:57.754788+0000","last_peered":"2024-09-11T01:21:57.754788+0000","last_clean":"2024-09-11T01:21:57.754788+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:57.754788+0000","last_undegraded":"2024-09-11T01:21:57.754788+0000","last_fullsized":"2024-09-11T01:21:57.754788+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19615744,"num_objects":4789,"num_object_clones":0,"num_object_copies":9578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15470","reported_seq":23814,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.589471+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:55.589471+0000","last_peered":"2024-09-11T01:21:55.589471+0000","last_clean":"2024-09-11T01:21:55.589471+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:55.589471+0000","last_undegraded":"2024-09-11T01:21:55.589471+0000","last_fullsized":"2024-09-11T01:21:55.589471+0000","mapping_epoch":40,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19575267,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13337,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15813","reported_seq":16092,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.872337+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:57.872337+0000","last_peered":"2024-09-11T01:21:57.872337+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:57.872337+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12177,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15359","reported_seq":18579,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.821740+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:55.821740+0000","last_peered":"2024-09-11T01:21:55.821740+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:55.821740+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579363,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13128,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15706","reported_seq":15860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.811576+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:55.811576+0000","last_peered":"2024-09-11T01:21:55.811576+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:55.811576+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19890176,"num_objects":4856,"num_object_clones":0,"num_object_copies":9712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15607","reported_seq":24076,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.615271+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:55.615271+0000","last_peered":"2024-09-11T01:21:55.615271+0000","last_clean":"2024-09-11T01:21:55.615271+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:55.615271+0000","last_undegraded":"2024-09-11T01:21:55.615271+0000","last_fullsized":"2024-09-11T01:21:55.615271+0000","mapping_epoch":42,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12297,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15901","reported_seq":16186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.808201+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:57.808201+0000","last_peered":"2024-09-11T01:21:57.808201+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:57.808201+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12294,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15827","reported_seq":16126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.867775+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:57.867775+0000","last_peered":"2024-09-11T01:21:57.867775+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:57.867775+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12362,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15494","reported_seq":15734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.788766+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:57.788766+0000","last_peered":"2024-09-11T01:21:57.788766+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:57.788766+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19022790,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12151,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":544},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":312025846,"num_objects":76188,"num_object_clones":0,"num_object_copies":152376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76188,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":209053,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10428,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829022208,"data_stored":828869544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160590,"ondisk_log_size":160590,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586052,"kb_used_data":287792,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93785788,"statfs":{"total":96636764160,"available":96036646912,"internally_reserved":0,"allocated":294699008,"data_stored":294417180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790032,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592156,"kb_used_data":231436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93779684,"statfs":{"total":96636764160,"available":96030396416,"internally_reserved":0,"allocated":236990464,"data_stored":236668429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699428,"kb_used_data":294344,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93672412,"statfs":{"total":96636764160,"available":95920549888,"internally_reserved":0,"allocated":301408256,"data_stored":301126423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":414772906},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299728896,"data_stored":299692766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":235532288,"data_stored":235455956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293761024,"data_stored":293720822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'15543", 'reported_seq': 15767, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.796620+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:21:57.796620+0000', 'last_peered': '2024-09-11T01:21:57.796620+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:21:57.796620+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'5500", 'ondisk_log_start': "39'5500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19230720, 'num_objects': 4695, 'num_object_clones': 0, 'num_object_copies': 9390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15543, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 647}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'15301", 'reported_seq': 15470, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:55.767539+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:21:55.767539+0000', 'last_peered': '2024-09-11T01:21:55.767539+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:21:55.767539+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'5300", 'ondisk_log_start': "39'5300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19431448, 'num_objects': 4745, 'num_object_clones': 0, 'num_object_copies': 9490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4745, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13362, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'15515", 'reported_seq': 15782, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.829664+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:21:57.829664+0000', 'last_peered': '2024-09-11T01:21:57.829664+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:21:57.829664+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'5500", 'ondisk_log_start': "39'5500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19218915, 'num_objects': 4693, 'num_object_clones': 0, 'num_object_copies': 9386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4693, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12003, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 644}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16032", 'reported_seq': 16320, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.807971+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:21:57.807971+0000', 'last_peered': '2024-09-11T01:21:57.807971+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:21:57.807971+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19579846, 'num_objects': 4782, 'num_object_clones': 0, 'num_object_copies': 9564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4782, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12785, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 576}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:00.701 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'15813", 'reported_seq': 16092, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.872337+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:21:57.872337+0000', 'last_peered': '2024-09-11T01:21:57.872337+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:21:57.872337+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'5800", 'ondisk_log_start': "39'5800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19582976, 'num_objects': 4781, 'num_object_clones': 0, 'num_object_copies': 9562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4781, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12177, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 751}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'15359", 'reported_seq': 18579, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:55.821740+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:21:55.821740+0000', 'last_peered': '2024-09-11T01:21:55.821740+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:21:55.821740+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'5300", 'ondisk_log_start': "39'5300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19579363, 'num_objects': 4781, 'num_object_clones': 0, 'num_object_copies': 9562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2466, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4781, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13128, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1562, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2315}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15706", 'reported_seq': 15860, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:55.811576+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:21:55.811576+0000', 'last_peered': '2024-09-11T01:21:55.811576+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:21:55.811576+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5700", 'ondisk_log_start': "39'5700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19890176, 'num_objects': 4856, 'num_object_clones': 0, 'num_object_copies': 9712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4856, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13343, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 709}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'15901", 'reported_seq': 16186, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.808201+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:21:57.808201+0000', 'last_peered': '2024-09-11T01:21:57.808201+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:21:57.808201+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19435520, 'num_objects': 4745, 'num_object_clones': 0, 'num_object_copies': 9490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12294, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 628}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'15827", 'reported_seq': 16126, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.867775+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:21:57.867775+0000', 'last_peered': '2024-09-11T01:21:57.867775+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:21:57.867775+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'5800", 'ondisk_log_start': "39'5800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19550208, 'num_objects': 4773, 'num_object_clones': 0, 'num_object_copies': 9546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12362, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:00.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'15494", 'reported_seq': 15734, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.788766+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:21:57.788766+0000', 'last_peered': '2024-09-11T01:21:57.788766+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:21:57.788766+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'5400", 'ondisk_log_start': "39'5400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19022790, 'num_objects': 4646, 'num_object_clones': 0, 'num_object_copies': 9292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4646, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12151, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 544}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:00.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:01.444 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:01.445 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39206/152380 objects degraded (25.729%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":521,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":76190,"data_bytes":312025865,"bytes_used":1922699264,"bytes_avail":287987593216,"bytes_total":289910292480,"degraded_objects":39206,"degraded_total":152380,"degraded_ratio":0.25729098306864417,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":383},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:01.474 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:01.474 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:02.055 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:02.056 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:02.078 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-11T01:22:00.587414+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":312025865,"num_objects":76190,"num_object_clones":0,"num_object_copies":152380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76190,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":209055,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10429,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160593,"ondisk_log_size":160593,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1877636,"kb_used_data":813572,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281237884,"statfs":{"total":289910292480,"available":287987593216,"internally_reserved":0,"allocated":833097728,"data_stored":832212032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1089522695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,5,11,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18853888,"num_objects":-4603,"num_object_clones":0,"num_object_copies":-9206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4603,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004820"},"pg_stats":[{"pgid":"2.f","version":"43'15543","reported_seq":15767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.796620+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:21:57.796620+0000","last_peered":"2024-09-11T01:21:57.796620+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:21:57.796620+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19230720,"num_objects":4695,"num_object_clones":0,"num_object_copies":9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15543,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15680","reported_seq":15761,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.856542+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:21:55.856542+0000","last_peered":"2024-09-11T01:21:55.856542+0000","last_clean":"2024-09-11T01:21:55.856542+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:21:55.856542+0000","last_undegraded":"2024-09-11T01:21:55.856542+0000","last_fullsized":"2024-09-11T01:21:55.856542+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19833798,"num_objects":4844,"num_object_clones":0,"num_object_copies":9688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4844,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13778,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15301","reported_seq":15470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.767539+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:21:55.767539+0000","last_peered":"2024-09-11T01:21:55.767539+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:21:55.767539+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431448,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13362,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15540","reported_seq":15586,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.827697+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:21:55.827697+0000","last_peered":"2024-09-11T01:21:55.827697+0000","last_clean":"2024-09-11T01:21:55.827697+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:21:55.827697+0000","last_undegraded":"2024-09-11T01:21:55.827697+0000","last_fullsized":"2024-09-11T01:21:55.827697+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15567","reported_seq":15689,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.834551+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:21:57.834551+0000","last_peered":"2024-09-11T01:21:57.834551+0000","last_clean":"2024-09-11T01:21:57.834551+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:21:57.834551+0000","last_undegraded":"2024-09-11T01:21:57.834551+0000","last_fullsized":"2024-09-11T01:21:57.834551+0000","mapping_epoch":31,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19292643,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12365,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15515","reported_seq":15782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.829664+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:21:57.829664+0000","last_peered":"2024-09-11T01:21:57.829664+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:21:57.829664+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19218915,"num_objects":4693,"num_object_clones":0,"num_object_copies":9386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12003,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":644},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16032","reported_seq":16320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.807971+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:21:57.807971+0000","last_peered":"2024-09-11T01:21:57.807971+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:21:57.807971+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579846,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12785,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":576},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15635","reported_seq":15755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.754788+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:21:57.754788+0000","last_peered":"2024-09-11T01:21:57.754788+0000","last_clean":"2024-09-11T01:21:57.754788+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:21:57.754788+0000","last_undegraded":"2024-09-11T01:21:57.754788+0000","last_fullsized":"2024-09-11T01:21:57.754788+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19615744,"num_objects":4789,"num_object_clones":0,"num_object_copies":9578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12588,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15470","reported_seq":23814,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.589471+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:21:55.589471+0000","last_peered":"2024-09-11T01:21:55.589471+0000","last_clean":"2024-09-11T01:21:55.589471+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:21:55.589471+0000","last_undegraded":"2024-09-11T01:21:55.589471+0000","last_fullsized":"2024-09-11T01:21:55.589471+0000","mapping_epoch":40,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19575267,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13337,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15813","reported_seq":16092,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.872337+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:21:57.872337+0000","last_peered":"2024-09-11T01:21:57.872337+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:21:57.872337+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12177,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15359","reported_seq":18579,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.821740+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:21:55.821740+0000","last_peered":"2024-09-11T01:21:55.821740+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:21:55.821740+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5300","ondisk_log_start":"39'5300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19579363,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2466,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13128,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2315}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15706","reported_seq":15860,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.811576+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:21:55.811576+0000","last_peered":"2024-09-11T01:21:55.811576+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:21:55.811576+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19890176,"num_objects":4856,"num_object_clones":0,"num_object_copies":9712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13343,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":709}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15607","reported_seq":24076,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.615271+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:21:55.615271+0000","last_peered":"2024-09-11T01:21:55.615271+0000","last_clean":"2024-09-11T01:21:55.615271+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:21:55.615271+0000","last_undegraded":"2024-09-11T01:21:55.615271+0000","last_fullsized":"2024-09-11T01:21:55.615271+0000","mapping_epoch":42,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12297,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'15901","reported_seq":16186,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.808201+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:21:57.808201+0000","last_peered":"2024-09-11T01:21:57.808201+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:21:57.808201+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12294,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":628}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15827","reported_seq":16126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.867775+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:21:57.867775+0000","last_peered":"2024-09-11T01:21:57.867775+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:21:57.867775+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12362,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15494","reported_seq":15734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.788766+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:21:57.788766+0000","last_peered":"2024-09-11T01:21:57.788766+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:21:57.788766+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19022790,"num_objects":4646,"num_object_clones":0,"num_object_copies":9292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4646,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12151,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":544},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":312025846,"num_objects":76188,"num_object_clones":0,"num_object_copies":152376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76188,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":209053,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10428,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829022208,"data_stored":828869544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160590,"ondisk_log_size":160590,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586052,"kb_used_data":287792,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93785788,"statfs":{"total":96636764160,"available":96036646912,"internally_reserved":0,"allocated":294699008,"data_stored":294417180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790032,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592156,"kb_used_data":231436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93779684,"statfs":{"total":96636764160,"available":96030396416,"internally_reserved":0,"allocated":236990464,"data_stored":236668429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699428,"kb_used_data":294344,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93672412,"statfs":{"total":96636764160,"available":95920549888,"internally_reserved":0,"allocated":301408256,"data_stored":301126423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":414772906},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,2,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":299728896,"data_stored":299692766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":235532288,"data_stored":235455956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293761024,"data_stored":293720822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:05.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:05.661 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:05.662 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:05.688 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-11T01:22:04.589047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":302678793,"num_objects":73908,"num_object_clones":0,"num_object_copies":147816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73908,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":211337,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10529,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1853796,"kb_used_data":793896,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281261724,"statfs":{"total":289910292480,"available":288012005376,"internally_reserved":0,"allocated":812949504,"data_stored":812063808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,5,9,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24178688,"num_objects":-5903,"num_object_clones":0,"num_object_copies":-11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005006"},"pg_stats":[{"pgid":"2.f","version":"43'15668","reported_seq":15887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.854498+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:02.854498+0000","last_peered":"2024-09-11T01:22:02.854498+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:02.854498+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15823","reported_seq":15904,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.778568+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:00.778568+0000","last_peered":"2024-09-11T01:22:00.778568+0000","last_clean":"2024-09-11T01:22:00.778568+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:00.778568+0000","last_undegraded":"2024-09-11T01:22:00.778568+0000","last_fullsized":"2024-09-11T01:22:00.778568+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19248070,"num_objects":4701,"num_object_clones":0,"num_object_copies":9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13921,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15451","reported_seq":15620,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.788267+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:00.788267+0000","last_peered":"2024-09-11T01:22:00.788267+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:00.788267+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18817048,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13512,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15680","reported_seq":15726,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.742856+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:00.742856+0000","last_peered":"2024-09-11T01:22:00.742856+0000","last_clean":"2024-09-11T01:22:00.742856+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:00.742856+0000","last_undegraded":"2024-09-11T01:22:00.742856+0000","last_fullsized":"2024-09-11T01:22:00.742856+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15711","reported_seq":15833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.837628+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:02.837628+0000","last_peered":"2024-09-11T01:22:02.837628+0000","last_clean":"2024-09-11T01:22:02.837628+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:02.837628+0000","last_undegraded":"2024-09-11T01:22:02.837628+0000","last_fullsized":"2024-09-11T01:22:02.837628+0000","mapping_epoch":31,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702819,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12509,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15670","reported_seq":15937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.829922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:02.829922+0000","last_peered":"2024-09-11T01:22:02.829922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:02.829922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18584035,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12158,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16158","reported_seq":16446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.775097+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:02.775097+0000","last_peered":"2024-09-11T01:22:02.775097+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:02.775097+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19063750,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12911,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":450},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15787","reported_seq":15907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.766829+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:02.766829+0000","last_peered":"2024-09-11T01:22:02.766829+0000","last_clean":"2024-09-11T01:22:02.766829+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:02.766829+0000","last_undegraded":"2024-09-11T01:22:02.766829+0000","last_fullsized":"2024-09-11T01:22:02.766829+0000","mapping_epoch":31,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12740,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15646","reported_seq":23989,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.596399+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:00.596399+0000","last_peered":"2024-09-11T01:22:00.596399+0000","last_clean":"2024-09-11T01:22:00.596399+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:00.596399+0000","last_undegraded":"2024-09-11T01:22:00.596399+0000","last_fullsized":"2024-09-11T01:22:00.596399+0000","mapping_epoch":40,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18854371,"num_objects":4604,"num_object_clones":0,"num_object_copies":9208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13513,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15955","reported_seq":16235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.831233+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:02.831233+0000","last_peered":"2024-09-11T01:22:02.831233+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:02.831233+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001344,"num_objects":4639,"num_object_clones":0,"num_object_copies":9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":609}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15502","reported_seq":18922,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.819361+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:00.819361+0000","last_peered":"2024-09-11T01:22:00.819361+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:00.819361+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993635,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13271,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2272}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15857","reported_seq":16011,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.800894+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:00.800894+0000","last_peered":"2024-09-11T01:22:00.800894+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:00.800894+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19271680,"num_objects":4705,"num_object_clones":0,"num_object_copies":9410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13494,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":558}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15738","reported_seq":24209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.567747+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:00.567747+0000","last_peered":"2024-09-11T01:22:00.567747+0000","last_clean":"2024-09-11T01:22:00.567747+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:00.567747+0000","last_undegraded":"2024-09-11T01:22:00.567747+0000","last_fullsized":"2024-09-11T01:22:00.567747+0000","mapping_epoch":42,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18898944,"num_objects":4614,"num_object_clones":0,"num_object_copies":9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12428,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16028","reported_seq":16313,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.863830+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:02.863830+0000","last_peered":"2024-09-11T01:22:02.863830+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:02.863830+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18915328,"num_objects":4618,"num_object_clones":0,"num_object_copies":9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12421,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15968","reported_seq":16268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.853941+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:02.853941+0000","last_peered":"2024-09-11T01:22:02.853941+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:02.853941+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15630","reported_seq":15870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.712699+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:02.712699+0000","last_peered":"2024-09-11T01:22:02.712699+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:02.712699+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18465734,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12287,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":408},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":302678774,"num_objects":73906,"num_object_clones":0,"num_object_copies":147812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73906,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":211335,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10528,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810110976,"data_stored":809958312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585420,"kb_used_data":283004,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93786420,"statfs":{"total":96636764160,"available":96037294080,"internally_reserved":0,"allocated":289796096,"data_stored":289514268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790033,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589308,"kb_used_data":224416,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93782532,"statfs":{"total":96636764160,"available":96033312768,"internally_reserved":0,"allocated":229801984,"data_stored":229479949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679068,"kb_used_data":286476,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93692772,"statfs":{"total":96636764160,"available":95941398528,"internally_reserved":0,"allocated":293351424,"data_stored":293069591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292909056,"data_stored":292872926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":228364288,"data_stored":228287956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288837632,"data_stored":288797430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:05.690 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:06.280 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:06.281 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:06.313 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-11T01:22:04.589047+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":302678793,"num_objects":73908,"num_object_clones":0,"num_object_copies":147816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73908,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":211337,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10529,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160775,"ondisk_log_size":160775,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1853796,"kb_used_data":793896,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281261724,"statfs":{"total":289910292480,"available":288012005376,"internally_reserved":0,"allocated":812949504,"data_stored":812063808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,5,9,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24178688,"num_objects":-5903,"num_object_clones":0,"num_object_copies":-11806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5903,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005006"},"pg_stats":[{"pgid":"2.f","version":"43'15668","reported_seq":15887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.854498+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:02.854498+0000","last_peered":"2024-09-11T01:22:02.854498+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:02.854498+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15823","reported_seq":15904,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.778568+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:00.778568+0000","last_peered":"2024-09-11T01:22:00.778568+0000","last_clean":"2024-09-11T01:22:00.778568+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:00.778568+0000","last_undegraded":"2024-09-11T01:22:00.778568+0000","last_fullsized":"2024-09-11T01:22:00.778568+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19248070,"num_objects":4701,"num_object_clones":0,"num_object_copies":9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13921,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15451","reported_seq":15620,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.788267+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:00.788267+0000","last_peered":"2024-09-11T01:22:00.788267+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:00.788267+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5400","ondisk_log_start":"39'5400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18817048,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13512,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15680","reported_seq":15726,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.742856+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:00.742856+0000","last_peered":"2024-09-11T01:22:00.742856+0000","last_clean":"2024-09-11T01:22:00.742856+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:00.742856+0000","last_undegraded":"2024-09-11T01:22:00.742856+0000","last_fullsized":"2024-09-11T01:22:00.742856+0000","mapping_epoch":31,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19177472,"num_objects":4682,"num_object_clones":0,"num_object_copies":9364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15711","reported_seq":15833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.837628+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:02.837628+0000","last_peered":"2024-09-11T01:22:02.837628+0000","last_clean":"2024-09-11T01:22:02.837628+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:02.837628+0000","last_undegraded":"2024-09-11T01:22:02.837628+0000","last_fullsized":"2024-09-11T01:22:02.837628+0000","mapping_epoch":31,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702819,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12509,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15670","reported_seq":15937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.829922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:02.829922+0000","last_peered":"2024-09-11T01:22:02.829922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:02.829922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18584035,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12158,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16158","reported_seq":16446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.775097+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:02.775097+0000","last_peered":"2024-09-11T01:22:02.775097+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:02.775097+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19063750,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12911,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":450},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15787","reported_seq":15907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.766829+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:02.766829+0000","last_peered":"2024-09-11T01:22:02.766829+0000","last_clean":"2024-09-11T01:22:02.766829+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:02.766829+0000","last_undegraded":"2024-09-11T01:22:02.766829+0000","last_fullsized":"2024-09-11T01:22:02.766829+0000","mapping_epoch":31,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12740,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15646","reported_seq":23989,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.596399+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:00.596399+0000","last_peered":"2024-09-11T01:22:00.596399+0000","last_clean":"2024-09-11T01:22:00.596399+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:00.596399+0000","last_undegraded":"2024-09-11T01:22:00.596399+0000","last_fullsized":"2024-09-11T01:22:00.596399+0000","mapping_epoch":40,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18854371,"num_objects":4604,"num_object_clones":0,"num_object_copies":9208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4604,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13513,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15955","reported_seq":16235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.831233+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:02.831233+0000","last_peered":"2024-09-11T01:22:02.831233+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:02.831233+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001344,"num_objects":4639,"num_object_clones":0,"num_object_copies":9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":609}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15502","reported_seq":18922,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.819361+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:00.819361+0000","last_peered":"2024-09-11T01:22:00.819361+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:00.819361+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5500","ondisk_log_start":"39'5500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993635,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13271,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1662,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2272}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'15857","reported_seq":16011,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.800894+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:00.800894+0000","last_peered":"2024-09-11T01:22:00.800894+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:00.800894+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19271680,"num_objects":4705,"num_object_clones":0,"num_object_copies":9410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13494,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":558}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15738","reported_seq":24209,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.567747+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:00.567747+0000","last_peered":"2024-09-11T01:22:00.567747+0000","last_clean":"2024-09-11T01:22:00.567747+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:00.567747+0000","last_undegraded":"2024-09-11T01:22:00.567747+0000","last_fullsized":"2024-09-11T01:22:00.567747+0000","mapping_epoch":42,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18898944,"num_objects":4614,"num_object_clones":0,"num_object_copies":9228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12428,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16028","reported_seq":16313,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.863830+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:02.863830+0000","last_peered":"2024-09-11T01:22:02.863830+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:02.863830+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18915328,"num_objects":4618,"num_object_clones":0,"num_object_copies":9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12421,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15968","reported_seq":16268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.853941+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:02.853941+0000","last_peered":"2024-09-11T01:22:02.853941+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:02.853941+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15630","reported_seq":15870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.712699+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:02.712699+0000","last_peered":"2024-09-11T01:22:02.712699+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:02.712699+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18465734,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12287,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":408},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":302678774,"num_objects":73906,"num_object_clones":0,"num_object_copies":147812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73906,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":211335,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10528,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":810110976,"data_stored":809958312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":585420,"kb_used_data":283004,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93786420,"statfs":{"total":96636764160,"available":96037294080,"internally_reserved":0,"allocated":289796096,"data_stored":289514268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790033,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589308,"kb_used_data":224416,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93782532,"statfs":{"total":96636764160,"available":96033312768,"internally_reserved":0,"allocated":229801984,"data_stored":229479949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679068,"kb_used_data":286476,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93692772,"statfs":{"total":96636764160,"available":95941398528,"internally_reserved":0,"allocated":293351424,"data_stored":293069591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292909056,"data_stored":292872926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":228364288,"data_stored":228287956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288837632,"data_stored":288797430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:06.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:06.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'15668", 'reported_seq': 15887, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.854498+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:02.854498+0000', 'last_peered': '2024-09-11T01:22:02.854498+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:02.854498+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18718720, 'num_objects': 4570, 'num_object_clones': 0, 'num_object_copies': 9140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15668, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 522}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:06.316 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'15451", 'reported_seq': 15620, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:00.788267+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:00.788267+0000', 'last_peered': '2024-09-11T01:22:00.788267+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:00.788267+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'5400", 'ondisk_log_start': "39'5400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18817048, 'num_objects': 4595, 'num_object_clones': 0, 'num_object_copies': 9190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4595, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13512, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 632}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:06.316 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:06.316 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'15670", 'reported_seq': 15937, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.829922+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:02.829922+0000', 'last_peered': '2024-09-11T01:22:02.829922+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:02.829922+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18584035, 'num_objects': 4538, 'num_object_clones': 0, 'num_object_copies': 9076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4538, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12158, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 489}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.316 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:06.316 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16158", 'reported_seq': 16446, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.775097+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:02.775097+0000', 'last_peered': '2024-09-11T01:22:02.775097+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:02.775097+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6100", 'ondisk_log_start': "39'6100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19063750, 'num_objects': 4656, 'num_object_clones': 0, 'num_object_copies': 9312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4656, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 12911, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 450}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:06.317 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'15955", 'reported_seq': 16235, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.831233+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:02.831233+0000', 'last_peered': '2024-09-11T01:22:02.831233+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:02.831233+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19001344, 'num_objects': 4639, 'num_object_clones': 0, 'num_object_copies': 9278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12319, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 609}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:06.317 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'15502", 'reported_seq': 18922, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:00.819361+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:00.819361+0000', 'last_peered': '2024-09-11T01:22:00.819361+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:00.819361+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'5500", 'ondisk_log_start': "39'5500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18993635, 'num_objects': 4638, 'num_object_clones': 0, 'num_object_copies': 9276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2366, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4638, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13271, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1662, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2272}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:06.317 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:06.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'15857", 'reported_seq': 16011, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:00.800894+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:00.800894+0000', 'last_peered': '2024-09-11T01:22:00.800894+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:00.800894+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'5800", 'ondisk_log_start': "39'5800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19271680, 'num_objects': 4705, 'num_object_clones': 0, 'num_object_copies': 9410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13494, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 558}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:06.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:06.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'16028", 'reported_seq': 16313, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.863830+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:02.863830+0000', 'last_peered': '2024-09-11T01:22:02.863830+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:02.863830+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18915328, 'num_objects': 4618, 'num_object_clones': 0, 'num_object_copies': 9236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4618, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12421, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 501}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.318 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:06.318 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'15968", 'reported_seq': 16268, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.853941+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:02.853941+0000', 'last_peered': '2024-09-11T01:22:02.853941+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:02.853941+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18972672, 'num_objects': 4632, 'num_object_clones': 0, 'num_object_copies': 9264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12503, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.319 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:06.319 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'15630", 'reported_seq': 15870, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.712699+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:02.712699+0000', 'last_peered': '2024-09-11T01:22:02.712699+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:02.712699+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18465734, 'num_objects': 4510, 'num_object_clones': 0, 'num_object_copies': 9020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4510, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12287, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 408}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:06.319 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:07.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:07.044 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39106/147816 objects degraded (26.456%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":527,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":73908,"data_bytes":302678793,"bytes_used":1898287104,"bytes_avail":288012005376,"bytes_total":289910292480,"degraded_objects":39106,"degraded_total":147816,"degraded_ratio":0.26455864047193811,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":491},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:16:18.435789+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:07.065 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:07.066 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:07.651 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:07.651 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:07.675 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-11T01:22:06.590804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298443529,"num_objects":72874,"num_object_clones":0,"num_object_copies":145748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72874,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":212371,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10638,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1837936,"kb_used_data":782204,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281277584,"statfs":{"total":289910292480,"available":288028246016,"internally_reserved":0,"allocated":800976896,"data_stored":800091200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1081003015},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,9,3,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22999040,"num_objects":-5615,"num_object_clones":0,"num_object_copies":-11230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5615,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006048"},"pg_stats":[{"pgid":"2.f","version":"43'15668","reported_seq":15887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.854498+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:02.854498+0000","last_peered":"2024-09-11T01:22:02.854498+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:02.854498+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18718720,"num_objects":4570,"num_object_clones":0,"num_object_copies":9140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15668,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15970","reported_seq":16049,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.869144+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:05.869144+0000","last_peered":"2024-09-11T01:22:05.869144+0000","last_clean":"2024-09-11T01:22:05.869144+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:05.869144+0000","last_undegraded":"2024-09-11T01:22:05.869144+0000","last_fullsized":"2024-09-11T01:22:05.869144+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18645958,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14068,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15607","reported_seq":15776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.893125+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:05.893125+0000","last_peered":"2024-09-11T01:22:05.893125+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:05.893125+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18178072,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13668,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":476}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15828","reported_seq":15874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.823815+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:05.823815+0000","last_peered":"2024-09-11T01:22:05.823815+0000","last_clean":"2024-09-11T01:22:05.823815+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:05.823815+0000","last_undegraded":"2024-09-11T01:22:05.823815+0000","last_fullsized":"2024-09-11T01:22:05.823815+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571264,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15711","reported_seq":15833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.837628+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:02.837628+0000","last_peered":"2024-09-11T01:22:02.837628+0000","last_clean":"2024-09-11T01:22:02.837628+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:02.837628+0000","last_undegraded":"2024-09-11T01:22:02.837628+0000","last_fullsized":"2024-09-11T01:22:02.837628+0000","mapping_epoch":31,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18702819,"num_objects":4567,"num_object_clones":0,"num_object_copies":9134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4567,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12509,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15670","reported_seq":15937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.829922+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:02.829922+0000","last_peered":"2024-09-11T01:22:02.829922+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:02.829922+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18584035,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12158,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16158","reported_seq":16446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.775097+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:02.775097+0000","last_peered":"2024-09-11T01:22:02.775097+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:02.775097+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19063750,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12911,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":450},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15787","reported_seq":15907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.766829+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:02.766829+0000","last_peered":"2024-09-11T01:22:02.766829+0000","last_clean":"2024-09-11T01:22:02.766829+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:02.766829+0000","last_undegraded":"2024-09-11T01:22:02.766829+0000","last_fullsized":"2024-09-11T01:22:02.766829+0000","mapping_epoch":31,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18993152,"num_objects":4637,"num_object_clones":0,"num_object_copies":9274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12740,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15791","reported_seq":24135,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.660791+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:05.660791+0000","last_peered":"2024-09-11T01:22:05.660791+0000","last_clean":"2024-09-11T01:22:05.660791+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:05.660791+0000","last_undegraded":"2024-09-11T01:22:05.660791+0000","last_fullsized":"2024-09-11T01:22:05.660791+0000","mapping_epoch":40,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18260451,"num_objects":4459,"num_object_clones":0,"num_object_copies":8918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13658,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'15955","reported_seq":16235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.831233+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:02.831233+0000","last_peered":"2024-09-11T01:22:02.831233+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:02.831233+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19001344,"num_objects":4639,"num_object_clones":0,"num_object_copies":9278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12319,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":609}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15639","reported_seq":19277,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.854542+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:05.854542+0000","last_peered":"2024-09-11T01:22:05.854542+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:05.854542+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18432483,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13408,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16023","reported_seq":16174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.844787+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:05.844787+0000","last_peered":"2024-09-11T01:22:05.844787+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:05.844787+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18591744,"num_objects":4539,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13660,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15873","reported_seq":24344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.735778+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:05.735778+0000","last_peered":"2024-09-11T01:22:05.735778+0000","last_clean":"2024-09-11T01:22:05.735778+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:05.735778+0000","last_undegraded":"2024-09-11T01:22:05.735778+0000","last_fullsized":"2024-09-11T01:22:05.735778+0000","mapping_epoch":42,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18345984,"num_objects":4479,"num_object_clones":0,"num_object_copies":8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16028","reported_seq":16313,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.863830+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:02.863830+0000","last_peered":"2024-09-11T01:22:02.863830+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:02.863830+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18915328,"num_objects":4618,"num_object_clones":0,"num_object_copies":9236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12421,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":501}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'15968","reported_seq":16268,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.853941+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:02.853941+0000","last_peered":"2024-09-11T01:22:02.853941+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:02.853941+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12503,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15630","reported_seq":15870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.712699+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:02.712699+0000","last_peered":"2024-09-11T01:22:02.712699+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:02.712699+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18465734,"num_objects":4510,"num_object_clones":0,"num_object_copies":9020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12287,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":408},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298443510,"num_objects":72872,"num_object_clones":0,"num_object_copies":145744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72872,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":212369,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10637,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798105600,"data_stored":797952936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160806,"ondisk_log_size":160806,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584912,"kb_used_data":278332,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93786928,"statfs":{"total":96636764160,"available":96037814272,"internally_reserved":0,"allocated":285011968,"data_stored":284730140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790034,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573956,"kb_used_data":217396,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93797884,"statfs":{"total":96636764160,"available":96049033216,"internally_reserved":0,"allocated":222613504,"data_stored":222291469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679068,"kb_used_data":286476,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93692772,"statfs":{"total":96636764160,"available":95941398528,"internally_reserved":0,"allocated":293351424,"data_stored":293069591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":401993386},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,1,4,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":292909056,"data_stored":292872926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221220864,"data_stored":221144532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283975680,"data_stored":283935478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:10.677 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:11.268 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:11.268 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:11.297 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-11T01:22:10.592503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293221129,"num_objects":71599,"num_object_clones":0,"num_object_copies":143198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71599,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":213646,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10638,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1836572,"kb_used_data":776680,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281278948,"statfs":{"total":289910292480,"available":288029642752,"internally_reserved":0,"allocated":795320320,"data_stored":794434624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,8,2,6,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18804736,"num_objects":-4591,"num_object_clones":0,"num_object_copies":-9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4591,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006031"},"pg_stats":[{"pgid":"2.f","version":"43'15810","reported_seq":16034,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.849386+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:07.849386+0000","last_peered":"2024-09-11T01:22:07.849386+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:07.849386+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15970","reported_seq":16049,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.869144+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:05.869144+0000","last_peered":"2024-09-11T01:22:05.869144+0000","last_clean":"2024-09-11T01:22:05.869144+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:05.869144+0000","last_undegraded":"2024-09-11T01:22:05.869144+0000","last_fullsized":"2024-09-11T01:22:05.869144+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18645958,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14068,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15607","reported_seq":15776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.893125+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:05.893125+0000","last_peered":"2024-09-11T01:22:05.893125+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:05.893125+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18178072,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13668,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":476}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15828","reported_seq":15874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.823815+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:05.823815+0000","last_peered":"2024-09-11T01:22:05.823815+0000","last_clean":"2024-09-11T01:22:05.823815+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:05.823815+0000","last_undegraded":"2024-09-11T01:22:05.823815+0000","last_fullsized":"2024-09-11T01:22:05.823815+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571264,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15854","reported_seq":15976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.878472+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:07.878472+0000","last_peered":"2024-09-11T01:22:07.878472+0000","last_clean":"2024-09-11T01:22:07.878472+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:07.878472+0000","last_undegraded":"2024-09-11T01:22:07.878472+0000","last_fullsized":"2024-09-11T01:22:07.878472+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18117091,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12652,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15817","reported_seq":16084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.817788+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:07.817788+0000","last_peered":"2024-09-11T01:22:07.817788+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:07.817788+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981923,"num_objects":4391,"num_object_clones":0,"num_object_copies":8782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12305,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":342},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16288","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.843375+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:07.843375+0000","last_peered":"2024-09-11T01:22:07.843375+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:07.843375+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18531270,"num_objects":4526,"num_object_clones":0,"num_object_copies":9052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13041,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":320},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15933","reported_seq":16053,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.852180+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:07.852180+0000","last_peered":"2024-09-11T01:22:07.852180+0000","last_clean":"2024-09-11T01:22:07.852180+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:07.852180+0000","last_undegraded":"2024-09-11T01:22:07.852180+0000","last_fullsized":"2024-09-11T01:22:07.852180+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18395136,"num_objects":4491,"num_object_clones":0,"num_object_copies":8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12886,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15791","reported_seq":24135,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.660791+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:05.660791+0000","last_peered":"2024-09-11T01:22:05.660791+0000","last_clean":"2024-09-11T01:22:05.660791+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:05.660791+0000","last_undegraded":"2024-09-11T01:22:05.660791+0000","last_fullsized":"2024-09-11T01:22:05.660791+0000","mapping_epoch":40,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18260451,"num_objects":4459,"num_object_clones":0,"num_object_copies":8918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13658,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16096","reported_seq":16373,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.874967+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:07.874967+0000","last_peered":"2024-09-11T01:22:07.874967+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:07.874967+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18423808,"num_objects":4498,"num_object_clones":0,"num_object_copies":8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12460,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15639","reported_seq":19277,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.854542+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:05.854542+0000","last_peered":"2024-09-11T01:22:05.854542+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:05.854542+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18432483,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13408,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16023","reported_seq":16174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.844787+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:05.844787+0000","last_peered":"2024-09-11T01:22:05.844787+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:05.844787+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18591744,"num_objects":4539,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13660,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15873","reported_seq":24344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.735778+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:05.735778+0000","last_peered":"2024-09-11T01:22:05.735778+0000","last_clean":"2024-09-11T01:22:05.735778+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:05.735778+0000","last_undegraded":"2024-09-11T01:22:05.735778+0000","last_fullsized":"2024-09-11T01:22:05.735778+0000","mapping_epoch":42,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18345984,"num_objects":4479,"num_object_clones":0,"num_object_copies":8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16163","reported_seq":16448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.834610+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:07.834610+0000","last_peered":"2024-09-11T01:22:07.834610+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:07.834610+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12556,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16104","reported_seq":16404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.835365+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:07.835365+0000","last_peered":"2024-09-11T01:22:07.835365+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:07.835365+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":4496,"num_object_clones":0,"num_object_copies":8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12639,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15785","reported_seq":16025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.788020+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:07.788020+0000","last_peered":"2024-09-11T01:22:07.788020+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:07.788020+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17830854,"num_objects":4355,"num_object_clones":0,"num_object_copies":8710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12442,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":253},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293221110,"num_objects":71597,"num_object_clones":0,"num_object_copies":143194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71597,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":213644,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10637,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791179264,"data_stored":791026600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584912,"kb_used_data":278332,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93786928,"statfs":{"total":96636764160,"available":96037814272,"internally_reserved":0,"allocated":285011968,"data_stored":284730140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790034,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573956,"kb_used_data":217396,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93797884,"statfs":{"total":96636764160,"available":96049033216,"internally_reserved":0,"allocated":222613504,"data_stored":222291469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677704,"kb_used_data":280952,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93694136,"statfs":{"total":96636764160,"available":95942795264,"internally_reserved":0,"allocated":287694848,"data_stored":287413015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285982720,"data_stored":285946590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221220864,"data_stored":221144532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283975680,"data_stored":283935478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:11.299 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:11.855 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:11.855 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:11.883 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-11T01:22:10.592503+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293221129,"num_objects":71599,"num_object_clones":0,"num_object_copies":143198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71599,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":213646,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10638,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1836572,"kb_used_data":776680,"kb_used_omap":12,"kb_used_meta":1059827,"kb_avail":281278948,"statfs":{"total":289910292480,"available":288029642752,"internally_reserved":0,"allocated":795320320,"data_stored":794434624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1085262855},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,8,2,6,9,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18804736,"num_objects":-4591,"num_object_clones":0,"num_object_copies":-9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4591,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006031"},"pg_stats":[{"pgid":"2.f","version":"43'15810","reported_seq":16034,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.849386+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:07.849386+0000","last_peered":"2024-09-11T01:22:07.849386+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:07.849386+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'15970","reported_seq":16049,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.869144+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:05.869144+0000","last_peered":"2024-09-11T01:22:05.869144+0000","last_clean":"2024-09-11T01:22:05.869144+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:05.869144+0000","last_undegraded":"2024-09-11T01:22:05.869144+0000","last_fullsized":"2024-09-11T01:22:05.869144+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18645958,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14068,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15607","reported_seq":15776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.893125+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:05.893125+0000","last_peered":"2024-09-11T01:22:05.893125+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:05.893125+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18178072,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13668,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":476}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15828","reported_seq":15874,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.823815+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:05.823815+0000","last_peered":"2024-09-11T01:22:05.823815+0000","last_clean":"2024-09-11T01:22:05.823815+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:05.823815+0000","last_undegraded":"2024-09-11T01:22:05.823815+0000","last_fullsized":"2024-09-11T01:22:05.823815+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18571264,"num_objects":4534,"num_object_clones":0,"num_object_copies":9068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15828,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15854","reported_seq":15976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.878472+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:07.878472+0000","last_peered":"2024-09-11T01:22:07.878472+0000","last_clean":"2024-09-11T01:22:07.878472+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:07.878472+0000","last_undegraded":"2024-09-11T01:22:07.878472+0000","last_fullsized":"2024-09-11T01:22:07.878472+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18117091,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12652,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15817","reported_seq":16084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.817788+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:07.817788+0000","last_peered":"2024-09-11T01:22:07.817788+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:07.817788+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981923,"num_objects":4391,"num_object_clones":0,"num_object_copies":8782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12305,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":342},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16288","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.843375+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:07.843375+0000","last_peered":"2024-09-11T01:22:07.843375+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:07.843375+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18531270,"num_objects":4526,"num_object_clones":0,"num_object_copies":9052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13041,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":320},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15933","reported_seq":16053,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.852180+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:07.852180+0000","last_peered":"2024-09-11T01:22:07.852180+0000","last_clean":"2024-09-11T01:22:07.852180+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:07.852180+0000","last_undegraded":"2024-09-11T01:22:07.852180+0000","last_fullsized":"2024-09-11T01:22:07.852180+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18395136,"num_objects":4491,"num_object_clones":0,"num_object_copies":8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12886,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15791","reported_seq":24135,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.660791+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:05.660791+0000","last_peered":"2024-09-11T01:22:05.660791+0000","last_clean":"2024-09-11T01:22:05.660791+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:05.660791+0000","last_undegraded":"2024-09-11T01:22:05.660791+0000","last_fullsized":"2024-09-11T01:22:05.660791+0000","mapping_epoch":40,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18260451,"num_objects":4459,"num_object_clones":0,"num_object_copies":8918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13658,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16096","reported_seq":16373,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.874967+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:07.874967+0000","last_peered":"2024-09-11T01:22:07.874967+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:07.874967+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18423808,"num_objects":4498,"num_object_clones":0,"num_object_copies":8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12460,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15639","reported_seq":19277,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.854542+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:05.854542+0000","last_peered":"2024-09-11T01:22:05.854542+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:05.854542+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5600","ondisk_log_start":"39'5600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18432483,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13408,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1771,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2244}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16023","reported_seq":16174,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.844787+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:05.844787+0000","last_peered":"2024-09-11T01:22:05.844787+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:05.844787+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18591744,"num_objects":4539,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13660,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'15873","reported_seq":24344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.735778+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:05.735778+0000","last_peered":"2024-09-11T01:22:05.735778+0000","last_clean":"2024-09-11T01:22:05.735778+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:05.735778+0000","last_undegraded":"2024-09-11T01:22:05.735778+0000","last_fullsized":"2024-09-11T01:22:05.735778+0000","mapping_epoch":42,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18345984,"num_objects":4479,"num_object_clones":0,"num_object_copies":8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16163","reported_seq":16448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.834610+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:07.834610+0000","last_peered":"2024-09-11T01:22:07.834610+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:07.834610+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12556,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16104","reported_seq":16404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.835365+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:07.835365+0000","last_peered":"2024-09-11T01:22:07.835365+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:07.835365+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":4496,"num_object_clones":0,"num_object_copies":8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12639,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15785","reported_seq":16025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.788020+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:07.788020+0000","last_peered":"2024-09-11T01:22:07.788020+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:07.788020+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17830854,"num_objects":4355,"num_object_clones":0,"num_object_copies":8710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12442,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":253},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293221110,"num_objects":71597,"num_object_clones":0,"num_object_copies":143194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71597,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":213644,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10637,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":791179264,"data_stored":791026600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584912,"kb_used_data":278332,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93786928,"statfs":{"total":96636764160,"available":96037814272,"internally_reserved":0,"allocated":285011968,"data_stored":284730140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790034,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573956,"kb_used_data":217396,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93797884,"statfs":{"total":96636764160,"available":96049033216,"internally_reserved":0,"allocated":222613504,"data_stored":222291469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677704,"kb_used_data":280952,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93694136,"statfs":{"total":96636764160,"available":95942795264,"internally_reserved":0,"allocated":287694848,"data_stored":287413015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285982720,"data_stored":285946590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":221220864,"data_stored":221144532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283975680,"data_stored":283935478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:11.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:11.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'15810", 'reported_seq': 16034, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.849386+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:07.849386+0000', 'last_peered': '2024-09-11T01:22:07.849386+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:07.849386+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'5800", 'ondisk_log_start': "39'5800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18137088, 'num_objects': 4428, 'num_object_clones': 0, 'num_object_copies': 8856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15810, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 380}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:11.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'15607", 'reported_seq': 15776, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:05.893125+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:05.893125+0000', 'last_peered': '2024-09-11T01:22:05.893125+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:05.893125+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18178072, 'num_objects': 4439, 'num_object_clones': 0, 'num_object_copies': 8878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4439, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13668, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 476}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:11.884 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:11.884 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'15817", 'reported_seq': 16084, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.817788+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:07.817788+0000', 'last_peered': '2024-09-11T01:22:07.817788+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:07.817788+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'5800", 'ondisk_log_start': "39'5800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17981923, 'num_objects': 4391, 'num_object_clones': 0, 'num_object_copies': 8782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4391, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12305, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 342}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16288", 'reported_seq': 16576, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.843375+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:07.843375+0000', 'last_peered': '2024-09-11T01:22:07.843375+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:07.843375+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6200", 'ondisk_log_start': "39'6200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18531270, 'num_objects': 4526, 'num_object_clones': 0, 'num_object_copies': 9052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4526, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13041, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 320}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'16096", 'reported_seq': 16373, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.874967+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:07.874967+0000', 'last_peered': '2024-09-11T01:22:07.874967+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:07.874967+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18423808, 'num_objects': 4498, 'num_object_clones': 0, 'num_object_copies': 8996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12460, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 468}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'15639", 'reported_seq': 19277, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:05.854542+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:05.854542+0000', 'last_peered': '2024-09-11T01:22:05.854542+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:05.854542+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'5600", 'ondisk_log_start': "39'5600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18432483, 'num_objects': 4501, 'num_object_clones': 0, 'num_object_copies': 9002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2257, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4501, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13408, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1771, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2244}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'16023", 'reported_seq': 16174, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:05.844787+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:05.844787+0000', 'last_peered': '2024-09-11T01:22:05.844787+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:05.844787+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18591744, 'num_objects': 4539, 'num_object_clones': 0, 'num_object_copies': 9078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4539, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13660, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 392}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'16163", 'reported_seq': 16448, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.834610+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:07.834610+0000', 'last_peered': '2024-09-11T01:22:07.834610+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:07.834610+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'6100", 'ondisk_log_start': "39'6100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18362368, 'num_objects': 4483, 'num_object_clones': 0, 'num_object_copies': 8966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4483, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12556, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 366}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:11.885 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'16104", 'reported_seq': 16404, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.835365+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:07.835365+0000', 'last_peered': '2024-09-11T01:22:07.835365+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:07.835365+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'6100", 'ondisk_log_start': "39'6100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18415616, 'num_objects': 4496, 'num_object_clones': 0, 'num_object_copies': 8992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12639, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 431}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.886 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:11.886 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'15785", 'reported_seq': 16025, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:07.788020+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:07.788020+0000', 'last_peered': '2024-09-11T01:22:07.788020+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:07.788020+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'5700", 'ondisk_log_start': "39'5700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17830854, 'num_objects': 4355, 'num_object_clones': 0, 'num_object_copies': 8710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4355, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12442, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 253}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:11.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:12.626 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:12.626 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38997/143198 objects degraded (27.233%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":533,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":71599,"data_bytes":293221129,"bytes_used":1880649728,"bytes_avail":288029642752,"bytes_total":289910292480,"degraded_objects":38997,"degraded_total":143198,"degraded_ratio":0.27232922247517422,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":382},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:12.652 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:12.652 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:13.246 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:13.246 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:13.267 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-09-11T01:22:12.593128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":289059593,"num_objects":70583,"num_object_clones":0,"num_object_copies":141166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70583,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":214662,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10729,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1851480,"kb_used_data":765220,"kb_used_omap":12,"kb_used_meta":1086195,"kb_avail":281264040,"statfs":{"total":289910292480,"available":288014376960,"internally_reserved":0,"allocated":783585280,"data_stored":782699584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1112263687},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,7,4,14,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22966272,"num_objects":-5607,"num_object_clones":0,"num_object_copies":-11214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5607,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005714"},"pg_stats":[{"pgid":"2.f","version":"43'15810","reported_seq":16034,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.849386+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:07.849386+0000","last_peered":"2024-09-11T01:22:07.849386+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:07.849386+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15810,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16103","reported_seq":16183,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.873907+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:10.873907+0000","last_peered":"2024-09-11T01:22:10.873907+0000","last_clean":"2024-09-11T01:22:10.873907+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:10.873907+0000","last_undegraded":"2024-09-11T01:22:10.873907+0000","last_fullsized":"2024-09-11T01:22:10.873907+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18101190,"num_objects":4421,"num_object_clones":0,"num_object_copies":8842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4421,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14201,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15771","reported_seq":15940,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.880560+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:10.880560+0000","last_peered":"2024-09-11T01:22:10.880560+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:10.880560+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17506328,"num_objects":4275,"num_object_clones":0,"num_object_copies":8550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4275,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13832,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":312}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'15975","reported_seq":16021,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.836332+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:10.836332+0000","last_peered":"2024-09-11T01:22:10.836332+0000","last_clean":"2024-09-11T01:22:10.836332+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:10.836332+0000","last_undegraded":"2024-09-11T01:22:10.836332+0000","last_fullsized":"2024-09-11T01:22:10.836332+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17969152,"num_objects":4387,"num_object_clones":0,"num_object_copies":8774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15975,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15854","reported_seq":15976,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.878472+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:07.878472+0000","last_peered":"2024-09-11T01:22:07.878472+0000","last_clean":"2024-09-11T01:22:07.878472+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:07.878472+0000","last_undegraded":"2024-09-11T01:22:07.878472+0000","last_fullsized":"2024-09-11T01:22:07.878472+0000","mapping_epoch":31,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18117091,"num_objects":4424,"num_object_clones":0,"num_object_copies":8848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12652,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15817","reported_seq":16084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.817788+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:07.817788+0000","last_peered":"2024-09-11T01:22:07.817788+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:07.817788+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5800","ondisk_log_start":"39'5800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981923,"num_objects":4391,"num_object_clones":0,"num_object_copies":8782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12305,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":342},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16288","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.843375+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:07.843375+0000","last_peered":"2024-09-11T01:22:07.843375+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:07.843375+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18531270,"num_objects":4526,"num_object_clones":0,"num_object_copies":9052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4526,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13041,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":320},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'15933","reported_seq":16053,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.852180+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:07.852180+0000","last_peered":"2024-09-11T01:22:07.852180+0000","last_clean":"2024-09-11T01:22:07.852180+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:07.852180+0000","last_undegraded":"2024-09-11T01:22:07.852180+0000","last_fullsized":"2024-09-11T01:22:07.852180+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18395136,"num_objects":4491,"num_object_clones":0,"num_object_copies":8982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12886,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'15926","reported_seq":24270,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.654668+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:10.654668+0000","last_peered":"2024-09-11T01:22:10.654668+0000","last_clean":"2024-09-11T01:22:10.654668+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:10.654668+0000","last_undegraded":"2024-09-11T01:22:10.654668+0000","last_fullsized":"2024-09-11T01:22:10.654668+0000","mapping_epoch":40,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17707491,"num_objects":4324,"num_object_clones":0,"num_object_copies":8648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13793,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16096","reported_seq":16373,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.874967+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:07.874967+0000","last_peered":"2024-09-11T01:22:07.874967+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:07.874967+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18423808,"num_objects":4498,"num_object_clones":0,"num_object_copies":8996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12460,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":468}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15792","reported_seq":19610,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.920509+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:10.920509+0000","last_peered":"2024-09-11T01:22:10.920509+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:10.920509+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17805795,"num_objects":4348,"num_object_clones":0,"num_object_copies":8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4348,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13561,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2182}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16153","reported_seq":16307,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.794481+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:10.794481+0000","last_peered":"2024-09-11T01:22:10.794481+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:10.794481+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18059264,"num_objects":4409,"num_object_clones":0,"num_object_copies":8818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13790,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16027","reported_seq":24498,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.782816+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:10.782816+0000","last_peered":"2024-09-11T01:22:10.782816+0000","last_clean":"2024-09-11T01:22:10.782816+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:10.782816+0000","last_undegraded":"2024-09-11T01:22:10.782816+0000","last_fullsized":"2024-09-11T01:22:10.782816+0000","mapping_epoch":42,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17715200,"num_objects":4325,"num_object_clones":0,"num_object_copies":8650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12717,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16163","reported_seq":16448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.834610+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:07.834610+0000","last_peered":"2024-09-11T01:22:07.834610+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:07.834610+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12556,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":366}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16104","reported_seq":16404,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.835365+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:07.835365+0000","last_peered":"2024-09-11T01:22:07.835365+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:07.835365+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18415616,"num_objects":4496,"num_object_clones":0,"num_object_copies":8992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12639,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":431}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15785","reported_seq":16025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:07.788020+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:07.788020+0000","last_peered":"2024-09-11T01:22:07.788020+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:07.788020+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5700","ondisk_log_start":"39'5700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17830854,"num_objects":4355,"num_object_clones":0,"num_object_copies":8710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12442,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":253},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":289059574,"num_objects":70581,"num_object_clones":0,"num_object_copies":141162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38906,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70581,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":214660,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10728,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":779149312,"data_stored":778996648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584424,"kb_used_data":273688,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93787416,"statfs":{"total":96636764160,"available":96038313984,"internally_reserved":0,"allocated":280256512,"data_stored":279974684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":318172846},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790035,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589352,"kb_used_data":210580,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93782488,"statfs":{"total":96636764160,"available":96033267712,"internally_reserved":0,"allocated":215633920,"data_stored":215311885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,2,6,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677704,"kb_used_data":280952,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93694136,"statfs":{"total":96636764160,"available":95942795264,"internally_reserved":0,"allocated":287694848,"data_stored":287413015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":406253226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,0,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285982720,"data_stored":285946590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214036480,"data_stored":213960148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279130112,"data_stored":279089910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:16.269 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:16.659 INFO:teuthology.orchestra.run.smithi039.stderr:2024-09-11T01:22:16.657+0000 7f2c71508640 0 --2- 172.21.15.39:0/546755576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2c6c08dcf0 0x7f2c6c12c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-11T01:22:17.044 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:17.045 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:17.077 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-11T01:22:16.594529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279487241,"num_objects":68246,"num_object_clones":0,"num_object_copies":136492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68246,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":216999,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10827,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1838348,"kb_used_data":746516,"kb_used_omap":12,"kb_used_meta":1091763,"kb_avail":281277172,"statfs":{"total":289910292480,"available":288027824128,"internally_reserved":0,"allocated":764432384,"data_stored":763546688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1117965319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,0,6,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23191552,"num_objects":-5662,"num_object_clones":0,"num_object_copies":-11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005481"},"pg_stats":[{"pgid":"2.f","version":"43'15955","reported_seq":16179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.930432+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:12.930432+0000","last_peered":"2024-09-11T01:22:12.930432+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:12.930432+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15955,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16253","reported_seq":16334,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.900689+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:15.900689+0000","last_peered":"2024-09-11T01:22:15.900689+0000","last_clean":"2024-09-11T01:22:15.900689+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:15.900689+0000","last_undegraded":"2024-09-11T01:22:15.900689+0000","last_fullsized":"2024-09-11T01:22:15.900689+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17486790,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14351,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15906","reported_seq":16075,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.894263+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:15.894263+0000","last_peered":"2024-09-11T01:22:15.894263+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:15.894263+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953368,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13967,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16117","reported_seq":16163,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.860067+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:15.860067+0000","last_peered":"2024-09-11T01:22:15.860067+0000","last_clean":"2024-09-11T01:22:15.860067+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:15.860067+0000","last_undegraded":"2024-09-11T01:22:15.860067+0000","last_fullsized":"2024-09-11T01:22:15.860067+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17387520,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15994","reported_seq":16116,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:12.895644+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:12.895644+0000","last_peered":"2024-09-11T01:22:12.895644+0000","last_clean":"2024-09-11T01:22:12.895644+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:12.895644+0000","last_undegraded":"2024-09-11T01:22:12.895644+0000","last_fullsized":"2024-09-11T01:22:12.895644+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543651,"num_objects":4284,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4284,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12792,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15985","reported_seq":16250,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.863497+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:12.863497+0000","last_peered":"2024-09-11T01:22:12.863497+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:12.863497+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17293795,"num_objects":4223,"num_object_clones":0,"num_object_copies":8446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12473,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16442","reported_seq":16730,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.839954+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:12.839954+0000","last_peered":"2024-09-11T01:22:12.839954+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:12.839954+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900486,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13195,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":166},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16093","reported_seq":16213,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:12.842113+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:12.842113+0000","last_peered":"2024-09-11T01:22:12.842113+0000","last_clean":"2024-09-11T01:22:12.842113+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:12.842113+0000","last_undegraded":"2024-09-11T01:22:12.842113+0000","last_fullsized":"2024-09-11T01:22:12.842113+0000","mapping_epoch":31,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17739776,"num_objects":4331,"num_object_clones":0,"num_object_copies":8662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13046,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16070","reported_seq":24414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.812606+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:15.812606+0000","last_peered":"2024-09-11T01:22:15.812606+0000","last_clean":"2024-09-11T01:22:15.812606+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:15.812606+0000","last_undegraded":"2024-09-11T01:22:15.812606+0000","last_fullsized":"2024-09-11T01:22:15.812606+0000","mapping_epoch":40,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117667,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13937,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16228","reported_seq":16505,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.921622+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:12.921622+0000","last_peered":"2024-09-11T01:22:12.921622+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:12.921622+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15949","reported_seq":19965,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.941169+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:15.941169+0000","last_peered":"2024-09-11T01:22:15.941169+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:15.941169+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162723,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13718,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1960,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16303","reported_seq":16457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.743689+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:15.743689+0000","last_peered":"2024-09-11T01:22:15.743689+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:15.743689+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13940,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":112}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16177","reported_seq":24647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.832401+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:15.832401+0000","last_peered":"2024-09-11T01:22:15.832401+0000","last_clean":"2024-09-11T01:22:15.832401+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:15.832401+0000","last_undegraded":"2024-09-11T01:22:15.832401+0000","last_fullsized":"2024-09-11T01:22:15.832401+0000","mapping_epoch":42,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16284","reported_seq":16569,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.903085+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:12.903085+0000","last_peered":"2024-09-11T01:22:12.903085+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:12.903085+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12677,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16260","reported_seq":16560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.919937+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:12.919937+0000","last_peered":"2024-09-11T01:22:12.919937+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:12.919937+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12795,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15918","reported_seq":16156,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.777737+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:12.777737+0000","last_peered":"2024-09-11T01:22:12.777737+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:12.777737+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17286086,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12575,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":120},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279487222,"num_objects":68244,"num_object_clones":0,"num_object_copies":136488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68244,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":216997,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10826,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":759959552,"data_stored":759806888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567252,"kb_used_data":268992,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93804588,"statfs":{"total":96636764160,"available":96055898112,"internally_reserved":0,"allocated":275447808,"data_stored":275165980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574196,"kb_used_data":203748,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93797644,"statfs":{"total":96636764160,"available":96048787456,"internally_reserved":0,"allocated":208637952,"data_stored":208315917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696900,"kb_used_data":273776,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93674940,"statfs":{"total":96636764160,"available":95923138560,"internally_reserved":0,"allocated":280346624,"data_stored":280064791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,6,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278749184,"data_stored":278713054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206946304,"data_stored":206869972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274264064,"data_stored":274223862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:17.079 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:17.672 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:17.673 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:17.702 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-11T01:22:16.594529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":279487241,"num_objects":68246,"num_object_clones":0,"num_object_copies":136492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68246,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":216999,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10827,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1838348,"kb_used_data":746516,"kb_used_omap":12,"kb_used_meta":1091763,"kb_avail":281277172,"statfs":{"total":289910292480,"available":288027824128,"internally_reserved":0,"allocated":764432384,"data_stored":763546688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1117965319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,0,6,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23191552,"num_objects":-5662,"num_object_clones":0,"num_object_copies":-11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-298,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005481"},"pg_stats":[{"pgid":"2.f","version":"43'15955","reported_seq":16179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.930432+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:12.930432+0000","last_peered":"2024-09-11T01:22:12.930432+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:12.930432+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543168,"num_objects":4283,"num_object_clones":0,"num_object_copies":8566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15955,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16253","reported_seq":16334,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.900689+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:15.900689+0000","last_peered":"2024-09-11T01:22:15.900689+0000","last_clean":"2024-09-11T01:22:15.900689+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:15.900689+0000","last_undegraded":"2024-09-11T01:22:15.900689+0000","last_fullsized":"2024-09-11T01:22:15.900689+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17486790,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14351,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15906","reported_seq":16075,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.894263+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:15.894263+0000","last_peered":"2024-09-11T01:22:15.894263+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:15.894263+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953368,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13967,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16117","reported_seq":16163,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.860067+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:15.860067+0000","last_peered":"2024-09-11T01:22:15.860067+0000","last_clean":"2024-09-11T01:22:15.860067+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:15.860067+0000","last_undegraded":"2024-09-11T01:22:15.860067+0000","last_fullsized":"2024-09-11T01:22:15.860067+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17387520,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'15994","reported_seq":16116,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:12.895644+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:12.895644+0000","last_peered":"2024-09-11T01:22:12.895644+0000","last_clean":"2024-09-11T01:22:12.895644+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:12.895644+0000","last_undegraded":"2024-09-11T01:22:12.895644+0000","last_fullsized":"2024-09-11T01:22:12.895644+0000","mapping_epoch":31,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17543651,"num_objects":4284,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4284,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12792,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'15985","reported_seq":16250,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.863497+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:12.863497+0000","last_peered":"2024-09-11T01:22:12.863497+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:12.863497+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17293795,"num_objects":4223,"num_object_clones":0,"num_object_copies":8446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12473,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":174},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16442","reported_seq":16730,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.839954+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:12.839954+0000","last_peered":"2024-09-11T01:22:12.839954+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:12.839954+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17900486,"num_objects":4372,"num_object_clones":0,"num_object_copies":8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4372,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13195,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":166},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16093","reported_seq":16213,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:12.842113+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:12.842113+0000","last_peered":"2024-09-11T01:22:12.842113+0000","last_clean":"2024-09-11T01:22:12.842113+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:12.842113+0000","last_undegraded":"2024-09-11T01:22:12.842113+0000","last_fullsized":"2024-09-11T01:22:12.842113+0000","mapping_epoch":31,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17739776,"num_objects":4331,"num_object_clones":0,"num_object_copies":8662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13046,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16070","reported_seq":24414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.812606+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:15.812606+0000","last_peered":"2024-09-11T01:22:15.812606+0000","last_clean":"2024-09-11T01:22:15.812606+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:15.812606+0000","last_undegraded":"2024-09-11T01:22:15.812606+0000","last_fullsized":"2024-09-11T01:22:15.812606+0000","mapping_epoch":40,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117667,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13937,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16228","reported_seq":16505,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.921622+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:12.921622+0000","last_peered":"2024-09-11T01:22:12.921622+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:12.921622+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17883136,"num_objects":4366,"num_object_clones":0,"num_object_copies":8732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12592,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15949","reported_seq":19965,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.941169+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:15.941169+0000","last_peered":"2024-09-11T01:22:15.941169+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:15.941169+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162723,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13718,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1960,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16303","reported_seq":16457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.743689+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:15.743689+0000","last_peered":"2024-09-11T01:22:15.743689+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:15.743689+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13940,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":112}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16177","reported_seq":24647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.832401+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:15.832401+0000","last_peered":"2024-09-11T01:22:15.832401+0000","last_clean":"2024-09-11T01:22:15.832401+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:15.832401+0000","last_undegraded":"2024-09-11T01:22:15.832401+0000","last_fullsized":"2024-09-11T01:22:15.832401+0000","mapping_epoch":42,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16284","reported_seq":16569,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.903085+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:12.903085+0000","last_peered":"2024-09-11T01:22:12.903085+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:12.903085+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17866752,"num_objects":4362,"num_object_clones":0,"num_object_copies":8724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12677,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":245}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16260","reported_seq":16560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.919937+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:12.919937+0000","last_peered":"2024-09-11T01:22:12.919937+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:12.919937+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12795,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'15918","reported_seq":16156,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:12.777737+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:12.777737+0000","last_peered":"2024-09-11T01:22:12.777737+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:12.777737+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17286086,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12575,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":120},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":279487222,"num_objects":68244,"num_object_clones":0,"num_object_copies":136488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68244,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":216997,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10826,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":759959552,"data_stored":759806888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567252,"kb_used_data":268992,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93804588,"statfs":{"total":96636764160,"available":96055898112,"internally_reserved":0,"allocated":275447808,"data_stored":275165980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574196,"kb_used_data":203748,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93797644,"statfs":{"total":96636764160,"available":96048787456,"internally_reserved":0,"allocated":208637952,"data_stored":208315917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696900,"kb_used_data":273776,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93674940,"statfs":{"total":96636764160,"available":95923138560,"internally_reserved":0,"allocated":280346624,"data_stored":280064791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,6,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":278749184,"data_stored":278713054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206946304,"data_stored":206869972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274264064,"data_stored":274223862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:17.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:17.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'15955", 'reported_seq': 16179, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.930432+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:12.930432+0000', 'last_peered': '2024-09-11T01:22:12.930432+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:12.930432+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17543168, 'num_objects': 4283, 'num_object_clones': 0, 'num_object_copies': 8566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15955, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 235}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:17.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'15906", 'reported_seq': 16075, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:15.894263+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:15.894263+0000', 'last_peered': '2024-09-11T01:22:15.894263+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:15.894263+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16953368, 'num_objects': 4140, 'num_object_clones': 0, 'num_object_copies': 8280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4140, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13967, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 177}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:17.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:17.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'15985", 'reported_seq': 16250, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.863497+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:12.863497+0000', 'last_peered': '2024-09-11T01:22:12.863497+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:12.863497+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17293795, 'num_objects': 4223, 'num_object_clones': 0, 'num_object_copies': 8446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4223, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12473, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 174}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16442", 'reported_seq': 16730, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.839954+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:12.839954+0000', 'last_peered': '2024-09-11T01:22:12.839954+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:12.839954+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6400", 'ondisk_log_start': "39'6400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17900486, 'num_objects': 4372, 'num_object_clones': 0, 'num_object_copies': 8744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4372, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13195, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 166}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'16228", 'reported_seq': 16505, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.921622+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:12.921622+0000', 'last_peered': '2024-09-11T01:22:12.921622+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:12.921622+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'6200", 'ondisk_log_start': "39'6200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17883136, 'num_objects': 4366, 'num_object_clones': 0, 'num_object_copies': 8732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12592, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 336}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'15949", 'reported_seq': 19965, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:15.941169+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:15.941169+0000', 'last_peered': '2024-09-11T01:22:15.941169+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:15.941169+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17162723, 'num_objects': 4191, 'num_object_clones': 0, 'num_object_copies': 8382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2068, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4191, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13718, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1960, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2123}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'16303", 'reported_seq': 16457, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:15.743689+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:15.743689+0000', 'last_peered': '2024-09-11T01:22:15.743689+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:15.743689+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17444864, 'num_objects': 4259, 'num_object_clones': 0, 'num_object_copies': 8518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13940, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 112}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'16284", 'reported_seq': 16569, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.903085+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:12.903085+0000', 'last_peered': '2024-09-11T01:22:12.903085+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:12.903085+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'6200", 'ondisk_log_start': "39'6200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17866752, 'num_objects': 4362, 'num_object_clones': 0, 'num_object_copies': 8724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4362, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12677, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:17.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'16260", 'reported_seq': 16560, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.919937+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:12.919937+0000', 'last_peered': '2024-09-11T01:22:12.919937+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:12.919937+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'6200", 'ondisk_log_start': "39'6200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17776640, 'num_objects': 4340, 'num_object_clones': 0, 'num_object_copies': 8680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4340, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12795, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 275}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:17.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'15918", 'reported_seq': 16156, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:12.777737+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:12.777737+0000', 'last_peered': '2024-09-11T01:22:12.777737+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:12.777737+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'5900", 'ondisk_log_start': "39'5900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17286086, 'num_objects': 4222, 'num_object_clones': 0, 'num_object_copies': 8444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4222, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12575, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 120}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:17.706 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:18.404 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:18.404 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38808/136492 objects degraded (28.432%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":538,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68246,"data_bytes":279487241,"bytes_used":1882468352,"bytes_avail":288027824128,"bytes_total":289910292480,"degraded_objects":38808,"degraded_total":136492,"degraded_ratio":0.28432435600621281,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":298,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":471},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:18.425 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:18.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:19.013 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:19.015 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:19.050 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":306,"stamp":"2024-09-11T01:22:18.595233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":274068233,"num_objects":66923,"num_object_clones":0,"num_object_copies":133846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66923,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":218322,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10827,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818836,"kb_used_data":739480,"kb_used_omap":12,"kb_used_meta":1079283,"kb_avail":281296684,"statfs":{"total":289910292480,"available":288047804416,"internally_reserved":0,"allocated":757227520,"data_stored":756341824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1105185799},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,13,0,6,6,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24375296,"num_objects":-5951,"num_object_clones":0,"num_object_copies":-11902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5951,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":189,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004429"},"pg_stats":[{"pgid":"2.f","version":"43'16092","reported_seq":16316,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.904110+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:17.904110+0000","last_peered":"2024-09-11T01:22:17.904110+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:17.904110+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16092,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16253","reported_seq":16334,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.900689+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:15.900689+0000","last_peered":"2024-09-11T01:22:15.900689+0000","last_clean":"2024-09-11T01:22:15.900689+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:15.900689+0000","last_undegraded":"2024-09-11T01:22:15.900689+0000","last_fullsized":"2024-09-11T01:22:15.900689+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17486790,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14351,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'15906","reported_seq":16075,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.894263+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:15.894263+0000","last_peered":"2024-09-11T01:22:15.894263+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:15.894263+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953368,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13967,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16117","reported_seq":16163,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.860067+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:15.860067+0000","last_peered":"2024-09-11T01:22:15.860067+0000","last_clean":"2024-09-11T01:22:15.860067+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:15.860067+0000","last_undegraded":"2024-09-11T01:22:15.860067+0000","last_fullsized":"2024-09-11T01:22:15.860067+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17387520,"num_objects":4245,"num_object_clones":0,"num_object_copies":8490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16117,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16160","reported_seq":16281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:17.963653+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:17.963653+0000","last_peered":"2024-09-11T01:22:17.963653+0000","last_clean":"2024-09-11T01:22:17.963653+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:17.963653+0000","last_undegraded":"2024-09-11T01:22:17.963653+0000","last_fullsized":"2024-09-11T01:22:17.963653+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863715,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12958,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16140","reported_seq":16407,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.842113+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:17.842113+0000","last_peered":"2024-09-11T01:22:17.842113+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:17.842113+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658915,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12628,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16581","reported_seq":16869,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.870995+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:17.870995+0000","last_peered":"2024-09-11T01:22:17.870995+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:17.870995+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17331142,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13334,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":27},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16249","reported_seq":16369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:17.866722+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:17.866722+0000","last_peered":"2024-09-11T01:22:17.866722+0000","last_clean":"2024-09-11T01:22:17.866722+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:17.866722+0000","last_undegraded":"2024-09-11T01:22:17.866722+0000","last_fullsized":"2024-09-11T01:22:17.866722+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13202,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16070","reported_seq":24414,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.812606+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:15.812606+0000","last_peered":"2024-09-11T01:22:15.812606+0000","last_clean":"2024-09-11T01:22:15.812606+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:15.812606+0000","last_undegraded":"2024-09-11T01:22:15.812606+0000","last_fullsized":"2024-09-11T01:22:15.812606+0000","mapping_epoch":40,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117667,"num_objects":4180,"num_object_clones":0,"num_object_copies":8360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13937,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16372","reported_seq":16650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.925210+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:17.925210+0000","last_peered":"2024-09-11T01:22:17.925210+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:17.925210+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17293312,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'15949","reported_seq":19965,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.941169+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:15.941169+0000","last_peered":"2024-09-11T01:22:15.941169+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:15.941169+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'5900","ondisk_log_start":"39'5900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17162723,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2068,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13718,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1960,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16303","reported_seq":16457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.743689+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:15.743689+0000","last_peered":"2024-09-11T01:22:15.743689+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:15.743689+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17444864,"num_objects":4259,"num_object_clones":0,"num_object_copies":8518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13940,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":112}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16177","reported_seq":24647,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.832401+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:15.832401+0000","last_peered":"2024-09-11T01:22:15.832401+0000","last_clean":"2024-09-11T01:22:15.832401+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:15.832401+0000","last_undegraded":"2024-09-11T01:22:15.832401+0000","last_fullsized":"2024-09-11T01:22:15.832401+0000","mapping_epoch":42,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16429","reported_seq":16714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.980046+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:17.980046+0000","last_peered":"2024-09-11T01:22:17.980046+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:17.980046+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272832,"num_objects":4217,"num_object_clones":0,"num_object_copies":8434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16393","reported_seq":16691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.002824+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:18.002824+0000","last_peered":"2024-09-11T01:22:18.002824+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:18.002824+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16066","reported_seq":16306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.985659+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:17.985659+0000","last_peered":"2024-09-11T01:22:17.985659+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:17.985659+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16679878,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12723,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-28},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":274068214,"num_objects":66921,"num_object_clones":0,"num_object_copies":133842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66921,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":218320,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10826,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":752824320,"data_stored":752671656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":567252,"kb_used_data":268992,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93804588,"statfs":{"total":96636764160,"available":96055898112,"internally_reserved":0,"allocated":275447808,"data_stored":275165980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":305393326},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790036,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574196,"kb_used_data":203748,"kb_used_omap":4,"kb_used_meta":370427,"kb_avail":93797644,"statfs":{"total":96636764160,"available":96048787456,"internally_reserved":0,"allocated":208637952,"data_stored":208315917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379317935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677388,"kb_used_data":266740,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93694452,"statfs":{"total":96636764160,"available":95943118848,"internally_reserved":0,"allocated":273141760,"data_stored":272859927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,8,0,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271613952,"data_stored":271577822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206946304,"data_stored":206869972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274264064,"data_stored":274223862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:22.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:22.637 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:22.640 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:22.665 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-11T01:22:22.596982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":269763337,"num_objects":65872,"num_object_clones":0,"num_object_copies":131744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65872,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":219373,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10926,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161011,"ondisk_log_size":161011,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818276,"kb_used_data":726428,"kb_used_omap":12,"kb_used_meta":1091763,"kb_avail":281297244,"statfs":{"total":289910292480,"available":288048377856,"internally_reserved":0,"allocated":743862272,"data_stored":742976576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1117965319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,8,2,6,14,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23457792,"num_objects":-5727,"num_object_clones":0,"num_object_copies":-11454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5727,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004479"},"pg_stats":[{"pgid":"2.f","version":"43'16092","reported_seq":16316,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.904110+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:17.904110+0000","last_peered":"2024-09-11T01:22:17.904110+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:17.904110+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16092,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16383","reported_seq":16464,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.702950+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:20.702950+0000","last_peered":"2024-09-11T01:22:20.702950+0000","last_clean":"2024-09-11T01:22:20.702950+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:20.702950+0000","last_undegraded":"2024-09-11T01:22:20.702950+0000","last_fullsized":"2024-09-11T01:22:20.702950+0000","mapping_epoch":31,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16954310,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16080","reported_seq":16248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.951146+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:20.951146+0000","last_peered":"2024-09-11T01:22:20.951146+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:20.951146+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16240664,"num_objects":3966,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14141,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16262","reported_seq":16308,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.942927+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:20.942927+0000","last_peered":"2024-09-11T01:22:20.942927+0000","last_clean":"2024-09-11T01:22:20.942927+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:20.942927+0000","last_undegraded":"2024-09-11T01:22:20.942927+0000","last_fullsized":"2024-09-11T01:22:20.942927+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16793600,"num_objects":4100,"num_object_clones":0,"num_object_copies":8200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16262,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16160","reported_seq":16281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:17.963653+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:17.963653+0000","last_peered":"2024-09-11T01:22:17.963653+0000","last_clean":"2024-09-11T01:22:17.963653+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:17.963653+0000","last_undegraded":"2024-09-11T01:22:17.963653+0000","last_fullsized":"2024-09-11T01:22:17.963653+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863715,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12958,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16140","reported_seq":16407,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.842113+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:17.842113+0000","last_peered":"2024-09-11T01:22:17.842113+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:17.842113+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658915,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12628,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16581","reported_seq":16869,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.870995+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:17.870995+0000","last_peered":"2024-09-11T01:22:17.870995+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:17.870995+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17331142,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13334,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":27},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16249","reported_seq":16369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:17.866722+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:17.866722+0000","last_peered":"2024-09-11T01:22:17.866722+0000","last_clean":"2024-09-11T01:22:17.866722+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:17.866722+0000","last_undegraded":"2024-09-11T01:22:17.866722+0000","last_fullsized":"2024-09-11T01:22:17.866722+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13202,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16235","reported_seq":24579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.848756+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:20.848756+0000","last_peered":"2024-09-11T01:22:20.848756+0000","last_clean":"2024-09-11T01:22:20.848756+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:20.848756+0000","last_undegraded":"2024-09-11T01:22:20.848756+0000","last_fullsized":"2024-09-11T01:22:20.848756+0000","mapping_epoch":40,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441827,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14102,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16372","reported_seq":16650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.925210+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:17.925210+0000","last_peered":"2024-09-11T01:22:17.925210+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:17.925210+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17293312,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16099","reported_seq":20313,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.939873+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:20.939873+0000","last_peered":"2024-09-11T01:22:20.939873+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:20.939873+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16548323,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13868,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2059,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2072}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16453","reported_seq":16607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.909102+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:20.909102+0000","last_peered":"2024-09-11T01:22:20.909102+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:20.909102+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16830464,"num_objects":4109,"num_object_clones":0,"num_object_copies":8218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14090,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-38}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16314","reported_seq":24785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.845686+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:20.845686+0000","last_peered":"2024-09-11T01:22:20.845686+0000","last_clean":"2024-09-11T01:22:20.845686+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:20.845686+0000","last_undegraded":"2024-09-11T01:22:20.845686+0000","last_fullsized":"2024-09-11T01:22:20.845686+0000","mapping_epoch":42,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16539648,"num_objects":4038,"num_object_clones":0,"num_object_copies":8076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13004,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16429","reported_seq":16714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.980046+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:17.980046+0000","last_peered":"2024-09-11T01:22:17.980046+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:17.980046+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272832,"num_objects":4217,"num_object_clones":0,"num_object_copies":8434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16393","reported_seq":16691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.002824+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:18.002824+0000","last_peered":"2024-09-11T01:22:18.002824+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:18.002824+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16066","reported_seq":16306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.985659+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:17.985659+0000","last_peered":"2024-09-11T01:22:17.985659+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:17.985659+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16679878,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12723,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-28},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":269763318,"num_objects":65870,"num_object_clones":0,"num_object_copies":131740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65870,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":219371,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10925,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":740757504,"data_stored":740604840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566720,"kb_used_data":264300,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93805120,"statfs":{"total":96636764160,"available":96056442880,"internally_reserved":0,"allocated":270643200,"data_stored":270361372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790037,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574168,"kb_used_data":195388,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93797672,"statfs":{"total":96636764160,"available":96048816128,"internally_reserved":0,"allocated":200077312,"data_stored":199755277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677388,"kb_used_data":266740,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93694452,"statfs":{"total":96636764160,"available":95943118848,"internally_reserved":0,"allocated":273141760,"data_stored":272859927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,8,0,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271613952,"data_stored":271577822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199688192,"data_stored":199611860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269455360,"data_stored":269415158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:22.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:23.261 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:23.263 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:23.283 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-11T01:22:22.596982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":269763337,"num_objects":65872,"num_object_clones":0,"num_object_copies":131744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65872,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":219373,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10926,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161011,"ondisk_log_size":161011,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818276,"kb_used_data":726428,"kb_used_omap":12,"kb_used_meta":1091763,"kb_avail":281297244,"statfs":{"total":289910292480,"available":288048377856,"internally_reserved":0,"allocated":743862272,"data_stored":742976576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1117965319},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,8,2,6,14,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23457792,"num_objects":-5727,"num_object_clones":0,"num_object_copies":-11454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5727,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004479"},"pg_stats":[{"pgid":"2.f","version":"43'16092","reported_seq":16316,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.904110+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:17.904110+0000","last_peered":"2024-09-11T01:22:17.904110+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:17.904110+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16092,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":98}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16383","reported_seq":16464,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.702950+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:20.702950+0000","last_peered":"2024-09-11T01:22:20.702950+0000","last_clean":"2024-09-11T01:22:20.702950+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:20.702950+0000","last_undegraded":"2024-09-11T01:22:20.702950+0000","last_fullsized":"2024-09-11T01:22:20.702950+0000","mapping_epoch":31,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16954310,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16080","reported_seq":16248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.951146+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:20.951146+0000","last_peered":"2024-09-11T01:22:20.951146+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:20.951146+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16240664,"num_objects":3966,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14141,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16262","reported_seq":16308,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.942927+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:20.942927+0000","last_peered":"2024-09-11T01:22:20.942927+0000","last_clean":"2024-09-11T01:22:20.942927+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:20.942927+0000","last_undegraded":"2024-09-11T01:22:20.942927+0000","last_fullsized":"2024-09-11T01:22:20.942927+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16793600,"num_objects":4100,"num_object_clones":0,"num_object_copies":8200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16262,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16160","reported_seq":16281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:17.963653+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:17.963653+0000","last_peered":"2024-09-11T01:22:17.963653+0000","last_clean":"2024-09-11T01:22:17.963653+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:17.963653+0000","last_undegraded":"2024-09-11T01:22:17.963653+0000","last_fullsized":"2024-09-11T01:22:17.963653+0000","mapping_epoch":31,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16863715,"num_objects":4118,"num_object_clones":0,"num_object_copies":8236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4118,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12958,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16140","reported_seq":16407,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.842113+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:17.842113+0000","last_peered":"2024-09-11T01:22:17.842113+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:17.842113+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6100","ondisk_log_start":"39'6100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16658915,"num_objects":4068,"num_object_clones":0,"num_object_copies":8136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4068,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12628,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":19},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16581","reported_seq":16869,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.870995+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:17.870995+0000","last_peered":"2024-09-11T01:22:17.870995+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:17.870995+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17331142,"num_objects":4233,"num_object_clones":0,"num_object_copies":8466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4233,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13334,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":27},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16249","reported_seq":16369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:17.866722+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:17.866722+0000","last_peered":"2024-09-11T01:22:17.866722+0000","last_clean":"2024-09-11T01:22:17.866722+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:17.866722+0000","last_undegraded":"2024-09-11T01:22:17.866722+0000","last_fullsized":"2024-09-11T01:22:17.866722+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17100800,"num_objects":4175,"num_object_clones":0,"num_object_copies":8350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13202,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16235","reported_seq":24579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.848756+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:20.848756+0000","last_peered":"2024-09-11T01:22:20.848756+0000","last_clean":"2024-09-11T01:22:20.848756+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:20.848756+0000","last_undegraded":"2024-09-11T01:22:20.848756+0000","last_fullsized":"2024-09-11T01:22:20.848756+0000","mapping_epoch":40,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441827,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14102,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16372","reported_seq":16650,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.925210+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:17.925210+0000","last_peered":"2024-09-11T01:22:17.925210+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:17.925210+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17293312,"num_objects":4222,"num_object_clones":0,"num_object_copies":8444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16099","reported_seq":20313,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.939873+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:20.939873+0000","last_peered":"2024-09-11T01:22:20.939873+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:20.939873+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16548323,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13868,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2059,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2072}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16453","reported_seq":16607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.909102+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:20.909102+0000","last_peered":"2024-09-11T01:22:20.909102+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:20.909102+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16830464,"num_objects":4109,"num_object_clones":0,"num_object_copies":8218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14090,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-38}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16314","reported_seq":24785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.845686+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:20.845686+0000","last_peered":"2024-09-11T01:22:20.845686+0000","last_clean":"2024-09-11T01:22:20.845686+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:20.845686+0000","last_undegraded":"2024-09-11T01:22:20.845686+0000","last_fullsized":"2024-09-11T01:22:20.845686+0000","mapping_epoch":42,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16539648,"num_objects":4038,"num_object_clones":0,"num_object_copies":8076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13004,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16429","reported_seq":16714,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.980046+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:17.980046+0000","last_peered":"2024-09-11T01:22:17.980046+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:17.980046+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17272832,"num_objects":4217,"num_object_clones":0,"num_object_copies":8434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":100}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16393","reported_seq":16691,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.002824+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:18.002824+0000","last_peered":"2024-09-11T01:22:18.002824+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:18.002824+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12928,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16066","reported_seq":16306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:17.985659+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:17.985659+0000","last_peered":"2024-09-11T01:22:17.985659+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:17.985659+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16679878,"num_objects":4074,"num_object_clones":0,"num_object_copies":8148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12723,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-28},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":269763318,"num_objects":65870,"num_object_clones":0,"num_object_copies":131740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65870,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":219371,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10925,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":740757504,"data_stored":740604840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566720,"kb_used_data":264300,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93805120,"statfs":{"total":96636764160,"available":96056442880,"internally_reserved":0,"allocated":270643200,"data_stored":270361372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790037,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574168,"kb_used_data":195388,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93797672,"statfs":{"total":96636764160,"available":96048816128,"internally_reserved":0,"allocated":200077312,"data_stored":199755277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677388,"kb_used_data":266740,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93694452,"statfs":{"total":96636764160,"available":95943118848,"internally_reserved":0,"allocated":273141760,"data_stored":272859927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,8,0,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":271613952,"data_stored":271577822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199688192,"data_stored":199611860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269455360,"data_stored":269415158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:23.289 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:23.289 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'16092", 'reported_seq': 16316, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:17.904110+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:17.904110+0000', 'last_peered': '2024-09-11T01:22:17.904110+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:17.904110+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16982016, 'num_objects': 4146, 'num_object_clones': 0, 'num_object_copies': 8292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16092, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 98}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.289 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:23.289 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'16080", 'reported_seq': 16248, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:20.951146+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:20.951146+0000', 'last_peered': '2024-09-11T01:22:20.951146+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:20.951146+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16240664, 'num_objects': 3966, 'num_object_clones': 0, 'num_object_copies': 7932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3966, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14141, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'16140", 'reported_seq': 16407, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:17.842113+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:17.842113+0000', 'last_peered': '2024-09-11T01:22:17.842113+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:17.842113+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'6100", 'ondisk_log_start': "39'6100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16658915, 'num_objects': 4068, 'num_object_clones': 0, 'num_object_copies': 8136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4068, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12628, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 19}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16581", 'reported_seq': 16869, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:17.870995+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:17.870995+0000', 'last_peered': '2024-09-11T01:22:17.870995+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:17.870995+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6500", 'ondisk_log_start': "39'6500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17331142, 'num_objects': 4233, 'num_object_clones': 0, 'num_object_copies': 8466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4233, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13334, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 27}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'16372", 'reported_seq': 16650, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:17.925210+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:17.925210+0000', 'last_peered': '2024-09-11T01:22:17.925210+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:17.925210+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17293312, 'num_objects': 4222, 'num_object_clones': 0, 'num_object_copies': 8444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12736, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 192}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'16099", 'reported_seq': 20313, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:20.939873+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:20.939873+0000', 'last_peered': '2024-09-11T01:22:20.939873+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:20.939873+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16548323, 'num_objects': 4041, 'num_object_clones': 0, 'num_object_copies': 8082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1969, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4041, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13868, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2059, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2072}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'16453", 'reported_seq': 16607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:20.909102+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:20.909102+0000', 'last_peered': '2024-09-11T01:22:20.909102+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:20.909102+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'6400", 'ondisk_log_start': "39'6400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16830464, 'num_objects': 4109, 'num_object_clones': 0, 'num_object_copies': 8218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14090, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -38}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:23.290 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'16429", 'reported_seq': 16714, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:17.980046+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:17.980046+0000', 'last_peered': '2024-09-11T01:22:17.980046+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:17.980046+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'6400", 'ondisk_log_start': "39'6400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17272832, 'num_objects': 4217, 'num_object_clones': 0, 'num_object_copies': 8434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12822, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 100}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.291 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:23.291 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'16393", 'reported_seq': 16691, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.002824+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:18.002824+0000', 'last_peered': '2024-09-11T01:22:18.002824+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:18.002824+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17231872, 'num_objects': 4207, 'num_object_clones': 0, 'num_object_copies': 8414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12928, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 142}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.291 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:23.291 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'16066", 'reported_seq': 16306, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:17.985659+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:17.985659+0000', 'last_peered': '2024-09-11T01:22:17.985659+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:17.985659+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'6000", 'ondisk_log_start': "39'6000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16679878, 'num_objects': 4074, 'num_object_clones': 0, 'num_object_copies': 8148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4074, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12723, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -28}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:23.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:24.006 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:24.007 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38709/131744 objects degraded (29.382%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":544,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65872,"data_bytes":269763337,"bytes_used":1861914624,"bytes_avail":288048377856,"bytes_total":289910292480,"degraded_objects":38709,"degraded_total":131744,"degraded_ratio":0.29381983240223464,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":288,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":477},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:24.026 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:24.026 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:24.618 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:24.620 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:24.641 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-11T01:22:24.597822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":264508169,"num_objects":64589,"num_object_clones":0,"num_object_copies":129178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64589,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":220656,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10926,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815676,"kb_used_data":719676,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281299844,"statfs":{"total":289910292480,"available":288051040256,"internally_reserved":0,"allocated":736948224,"data_stored":736062528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1122225159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7,18,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24551424,"num_objects":-5994,"num_object_clones":0,"num_object_copies":-11988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5994,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004693"},"pg_stats":[{"pgid":"2.f","version":"43'16231","reported_seq":16455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.916151+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:22.916151+0000","last_peered":"2024-09-11T01:22:22.916151+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:22.916151+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16412672,"num_objects":4007,"num_object_clones":0,"num_object_copies":8014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16231,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-41}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16383","reported_seq":16464,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.702950+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:20.702950+0000","last_peered":"2024-09-11T01:22:20.702950+0000","last_clean":"2024-09-11T01:22:20.702950+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:20.702950+0000","last_undegraded":"2024-09-11T01:22:20.702950+0000","last_fullsized":"2024-09-11T01:22:20.702950+0000","mapping_epoch":31,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16954310,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14481,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16080","reported_seq":16248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.951146+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:20.951146+0000","last_peered":"2024-09-11T01:22:20.951146+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:20.951146+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16240664,"num_objects":3966,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3966,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14141,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16262","reported_seq":16308,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.942927+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:20.942927+0000","last_peered":"2024-09-11T01:22:20.942927+0000","last_clean":"2024-09-11T01:22:20.942927+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:20.942927+0000","last_undegraded":"2024-09-11T01:22:20.942927+0000","last_fullsized":"2024-09-11T01:22:20.942927+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16793600,"num_objects":4100,"num_object_clones":0,"num_object_copies":8200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16262,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16290","reported_seq":16412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:22.905875+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:22.905875+0000","last_peered":"2024-09-11T01:22:22.905875+0000","last_clean":"2024-09-11T01:22:22.905875+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:22.905875+0000","last_undegraded":"2024-09-11T01:22:22.905875+0000","last_fullsized":"2024-09-11T01:22:22.905875+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16331235,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13088,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16275","reported_seq":16542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.945227+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:22.945227+0000","last_peered":"2024-09-11T01:22:22.945227+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:22.945227+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16105955,"num_objects":3933,"num_object_clones":0,"num_object_copies":7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3933,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12763,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-116},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16719","reported_seq":17006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.980758+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:22.980758+0000","last_peered":"2024-09-11T01:22:22.980758+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:22.980758+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16765894,"num_objects":4095,"num_object_clones":0,"num_object_copies":8190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4095,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13472,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-111},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16389","reported_seq":16509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:23.006051+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:23.006051+0000","last_peered":"2024-09-11T01:22:23.006051+0000","last_clean":"2024-09-11T01:22:23.006051+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:23.006051+0000","last_undegraded":"2024-09-11T01:22:23.006051+0000","last_fullsized":"2024-09-11T01:22:23.006051+0000","mapping_epoch":31,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13342,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16235","reported_seq":24579,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.848756+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:20.848756+0000","last_peered":"2024-09-11T01:22:20.848756+0000","last_clean":"2024-09-11T01:22:20.848756+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:20.848756+0000","last_undegraded":"2024-09-11T01:22:20.848756+0000","last_fullsized":"2024-09-11T01:22:20.848756+0000","mapping_epoch":40,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16441827,"num_objects":4015,"num_object_clones":0,"num_object_copies":8030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4015,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14102,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16528","reported_seq":16808,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.794598+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:22.794598+0000","last_peered":"2024-09-11T01:22:22.794598+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:22.794598+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16654336,"num_objects":4066,"num_object_clones":0,"num_object_copies":8132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12892,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16099","reported_seq":20313,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.939873+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:20.939873+0000","last_peered":"2024-09-11T01:22:20.939873+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:20.939873+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6000","ondisk_log_start":"39'6000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16548323,"num_objects":4041,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1969,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4041,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13868,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2059,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2072}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16453","reported_seq":16607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:20.909102+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:20.909102+0000","last_peered":"2024-09-11T01:22:20.909102+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:20.909102+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16830464,"num_objects":4109,"num_object_clones":0,"num_object_copies":8218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14090,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-38}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16314","reported_seq":24785,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:20.845686+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:20.845686+0000","last_peered":"2024-09-11T01:22:20.845686+0000","last_clean":"2024-09-11T01:22:20.845686+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:20.845686+0000","last_undegraded":"2024-09-11T01:22:20.845686+0000","last_fullsized":"2024-09-11T01:22:20.845686+0000","mapping_epoch":42,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16539648,"num_objects":4038,"num_object_clones":0,"num_object_copies":8076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13004,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16590","reported_seq":16875,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.975230+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:22.975230+0000","last_peered":"2024-09-11T01:22:22.975230+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:22.975230+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16613376,"num_objects":4056,"num_object_clones":0,"num_object_copies":8112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12983,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-61}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16531","reported_seq":16831,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:23.004858+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:23.004858+0000","last_peered":"2024-09-11T01:22:23.004858+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:23.004858+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13066,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16212","reported_seq":16452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.990134+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:22.990134+0000","last_peered":"2024-09-11T01:22:22.990134+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:22.990134+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16081862,"num_objects":3928,"num_object_clones":0,"num_object_copies":7856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3928,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12869,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-174},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":264508150,"num_objects":64587,"num_object_clones":0,"num_object_copies":129174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64587,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":220654,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10925,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":733732864,"data_stored":733580200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566720,"kb_used_data":264300,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93805120,"statfs":{"total":96636764160,"available":96056442880,"internally_reserved":0,"allocated":270643200,"data_stored":270361372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":309653166},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790037,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574168,"kb_used_data":195388,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93797672,"statfs":{"total":96636764160,"available":96048816128,"internally_reserved":0,"allocated":200077312,"data_stored":199755277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674788,"kb_used_data":259988,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93697052,"statfs":{"total":96636764160,"available":95945781248,"internally_reserved":0,"allocated":266227712,"data_stored":265945879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264589312,"data_stored":264553182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199688192,"data_stored":199611860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269455360,"data_stored":269415158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:27.643 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:28.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:28.273 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:28.303 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-11T01:22:26.598675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":260289289,"num_objects":63559,"num_object_clones":0,"num_object_copies":127118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63559,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":221686,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11020,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160724,"ondisk_log_size":160724,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1812576,"kb_used_data":708264,"kb_used_omap":12,"kb_used_meta":1104243,"kb_avail":281302944,"statfs":{"total":289910292480,"available":288054214656,"internally_reserved":0,"allocated":725262336,"data_stored":724376640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1130744839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,9,19,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23408640,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004810"},"pg_stats":[{"pgid":"2.f","version":"43'16231","reported_seq":16455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.916151+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:22.916151+0000","last_peered":"2024-09-11T01:22:22.916151+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:22.916151+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16412672,"num_objects":4007,"num_object_clones":0,"num_object_copies":8014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16231,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-41}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16528","reported_seq":16607,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.936801+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:25.936801+0000","last_peered":"2024-09-11T01:22:25.936801+0000","last_clean":"2024-09-11T01:22:25.936801+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:25.936801+0000","last_undegraded":"2024-09-11T01:22:25.936801+0000","last_fullsized":"2024-09-11T01:22:25.936801+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16360390,"num_objects":3996,"num_object_clones":0,"num_object_copies":7992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3996,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14626,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16220","reported_seq":16388,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.938978+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:25.938978+0000","last_peered":"2024-09-11T01:22:25.938978+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:25.938978+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15667224,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14281,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16411","reported_seq":16457,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.938143+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:25.938143+0000","last_peered":"2024-09-11T01:22:25.938143+0000","last_clean":"2024-09-11T01:22:25.938143+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:25.938143+0000","last_undegraded":"2024-09-11T01:22:25.938143+0000","last_fullsized":"2024-09-11T01:22:25.938143+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16290","reported_seq":16412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:22.905875+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:22.905875+0000","last_peered":"2024-09-11T01:22:22.905875+0000","last_clean":"2024-09-11T01:22:22.905875+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:22.905875+0000","last_undegraded":"2024-09-11T01:22:22.905875+0000","last_fullsized":"2024-09-11T01:22:22.905875+0000","mapping_epoch":31,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16331235,"num_objects":3988,"num_object_clones":0,"num_object_copies":7976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13088,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16275","reported_seq":16542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.945227+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:22.945227+0000","last_peered":"2024-09-11T01:22:22.945227+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:22.945227+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16105955,"num_objects":3933,"num_object_clones":0,"num_object_copies":7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3933,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12763,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-116},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16719","reported_seq":17006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.980758+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:22.980758+0000","last_peered":"2024-09-11T01:22:22.980758+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:22.980758+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16765894,"num_objects":4095,"num_object_clones":0,"num_object_copies":8190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4095,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13472,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-111},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16389","reported_seq":16509,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:23.006051+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:23.006051+0000","last_peered":"2024-09-11T01:22:23.006051+0000","last_clean":"2024-09-11T01:22:23.006051+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:23.006051+0000","last_undegraded":"2024-09-11T01:22:23.006051+0000","last_fullsized":"2024-09-11T01:22:23.006051+0000","mapping_epoch":31,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16527360,"num_objects":4035,"num_object_clones":0,"num_object_copies":8070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13342,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16381","reported_seq":24725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.935925+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:25.935925+0000","last_peered":"2024-09-11T01:22:25.935925+0000","last_clean":"2024-09-11T01:22:25.935925+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:25.935925+0000","last_undegraded":"2024-09-11T01:22:25.935925+0000","last_fullsized":"2024-09-11T01:22:25.935925+0000","mapping_epoch":40,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843811,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14248,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16528","reported_seq":16808,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.794598+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:22.794598+0000","last_peered":"2024-09-11T01:22:22.794598+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:22.794598+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16654336,"num_objects":4066,"num_object_clones":0,"num_object_copies":8132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12892,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":36}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16238","reported_seq":20640,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.929744+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:25.929744+0000","last_peered":"2024-09-11T01:22:25.929744+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:25.929744+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978979,"num_objects":3902,"num_object_clones":0,"num_object_copies":7804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14007,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2153,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16608","reported_seq":16762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.936478+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:25.936478+0000","last_peered":"2024-09-11T01:22:25.936478+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:25.936478+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14245,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-193}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16470","reported_seq":24941,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.848476+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:25.848476+0000","last_peered":"2024-09-11T01:22:25.848476+0000","last_clean":"2024-09-11T01:22:25.848476+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:25.848476+0000","last_undegraded":"2024-09-11T01:22:25.848476+0000","last_fullsized":"2024-09-11T01:22:25.848476+0000","mapping_epoch":42,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900672,"num_objects":3882,"num_object_clones":0,"num_object_copies":7764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13160,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16590","reported_seq":16875,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.975230+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:22.975230+0000","last_peered":"2024-09-11T01:22:22.975230+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:22.975230+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16613376,"num_objects":4056,"num_object_clones":0,"num_object_copies":8112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12983,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-61}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16531","reported_seq":16831,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:23.004858+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:23.004858+0000","last_peered":"2024-09-11T01:22:23.004858+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:23.004858+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16666624,"num_objects":4069,"num_object_clones":0,"num_object_copies":8138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13066,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16212","reported_seq":16452,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:22.990134+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:22.990134+0000","last_peered":"2024-09-11T01:22:22.990134+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:22.990134+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16081862,"num_objects":3928,"num_object_clones":0,"num_object_copies":7856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3928,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12869,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-174},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":260289270,"num_objects":63557,"num_object_clones":0,"num_object_copies":127114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63557,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":221684,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11019,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":721911808,"data_stored":721759144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160721,"ondisk_log_size":160721,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566416,"kb_used_data":259832,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93805424,"statfs":{"total":96636764160,"available":96056754176,"internally_reserved":0,"allocated":266067968,"data_stored":265786140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790038,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571372,"kb_used_data":188444,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93800468,"statfs":{"total":96636764160,"available":96051679232,"internally_reserved":0,"allocated":192966656,"data_stored":192644621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392097455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,5,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":674788,"kb_used_data":259988,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93697052,"statfs":{"total":96636764160,"available":95945781248,"internally_reserved":0,"allocated":266227712,"data_stored":265945879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":264589312,"data_stored":264553182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192536576,"data_stored":192460244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264785920,"data_stored":264745718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:28.305 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:28.887 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:28.890 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:28.911 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-11T01:22:28.599583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":254903049,"num_objects":62244,"num_object_clones":0,"num_object_copies":124488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62244,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":223001,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11020,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1813920,"kb_used_data":701284,"kb_used_omap":12,"kb_used_meta":1112563,"kb_avail":281301600,"statfs":{"total":289910292480,"available":288052838400,"internally_reserved":0,"allocated":718114816,"data_stored":717229120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1139264519},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,7,7,16,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24584192,"num_objects":-6002,"num_object_clones":0,"num_object_copies":-12004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6002,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005053"},"pg_stats":[{"pgid":"2.f","version":"43'16360","reported_seq":16584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.016520+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:28.016520+0000","last_peered":"2024-09-11T01:22:28.016520+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:28.016520+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15884288,"num_objects":3878,"num_object_clones":0,"num_object_copies":7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16528","reported_seq":16607,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.936801+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:25.936801+0000","last_peered":"2024-09-11T01:22:25.936801+0000","last_clean":"2024-09-11T01:22:25.936801+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:25.936801+0000","last_undegraded":"2024-09-11T01:22:25.936801+0000","last_fullsized":"2024-09-11T01:22:25.936801+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16360390,"num_objects":3996,"num_object_clones":0,"num_object_copies":7992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3996,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14626,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16220","reported_seq":16388,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.938978+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:25.938978+0000","last_peered":"2024-09-11T01:22:25.938978+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:25.938978+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15667224,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14281,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16411","reported_seq":16457,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.938143+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:25.938143+0000","last_peered":"2024-09-11T01:22:25.938143+0000","last_clean":"2024-09-11T01:22:25.938143+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:25.938143+0000","last_undegraded":"2024-09-11T01:22:25.938143+0000","last_fullsized":"2024-09-11T01:22:25.938143+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16446","reported_seq":16568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:27.991265+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:27.991265+0000","last_peered":"2024-09-11T01:22:27.991265+0000","last_clean":"2024-09-11T01:22:27.991265+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:27.991265+0000","last_undegraded":"2024-09-11T01:22:27.991265+0000","last_fullsized":"2024-09-11T01:22:27.991265+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16435","reported_seq":16702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.990868+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:27.990868+0000","last_peered":"2024-09-11T01:22:27.990868+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:27.990868+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450595,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12923,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-276},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16888","reported_seq":17176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.978815+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:27.978815+0000","last_peered":"2024-09-11T01:22:27.978815+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:27.978815+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16073670,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13641,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-280},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16523","reported_seq":16642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:28.026251+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:28.026251+0000","last_peered":"2024-09-11T01:22:28.026251+0000","last_clean":"2024-09-11T01:22:28.026251+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:28.026251+0000","last_undegraded":"2024-09-11T01:22:28.026251+0000","last_fullsized":"2024-09-11T01:22:28.026251+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16381","reported_seq":24725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.935925+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:25.935925+0000","last_peered":"2024-09-11T01:22:25.935925+0000","last_clean":"2024-09-11T01:22:25.935925+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:25.935925+0000","last_undegraded":"2024-09-11T01:22:25.935925+0000","last_fullsized":"2024-09-11T01:22:25.935925+0000","mapping_epoch":40,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843811,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14248,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16668","reported_seq":16947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.015866+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:28.015866+0000","last_peered":"2024-09-11T01:22:28.015866+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:28.015866+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13032,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16238","reported_seq":20640,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.929744+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:25.929744+0000","last_peered":"2024-09-11T01:22:25.929744+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:25.929744+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978979,"num_objects":3902,"num_object_clones":0,"num_object_copies":7804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14007,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2153,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16608","reported_seq":16762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.936478+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:25.936478+0000","last_peered":"2024-09-11T01:22:25.936478+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:25.936478+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14245,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-193}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16470","reported_seq":24941,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.848476+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:25.848476+0000","last_peered":"2024-09-11T01:22:25.848476+0000","last_clean":"2024-09-11T01:22:25.848476+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:25.848476+0000","last_undegraded":"2024-09-11T01:22:25.848476+0000","last_fullsized":"2024-09-11T01:22:25.848476+0000","mapping_epoch":42,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900672,"num_objects":3882,"num_object_clones":0,"num_object_copies":7764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13160,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16730","reported_seq":17015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.964161+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:27.964161+0000","last_peered":"2024-09-11T01:22:27.964161+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:27.964161+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16039936,"num_objects":3916,"num_object_clones":0,"num_object_copies":7832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13123,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16694","reported_seq":16994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.998755+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:27.998755+0000","last_peered":"2024-09-11T01:22:27.998755+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:27.998755+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16336","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.933128+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:27.933128+0000","last_peered":"2024-09-11T01:22:27.933128+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:27.933128+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15573958,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12993,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":254903030,"num_objects":62242,"num_object_clones":0,"num_object_copies":124484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62242,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":222999,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11019,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":714649600,"data_stored":714496936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566416,"kb_used_data":259832,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93805424,"statfs":{"total":96636764160,"available":96056754176,"internally_reserved":0,"allocated":266067968,"data_stored":265786140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790038,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571372,"kb_used_data":188444,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93800468,"statfs":{"total":96636764160,"available":96051679232,"internally_reserved":0,"allocated":192966656,"data_stored":192644621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392097455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,5,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676132,"kb_used_data":253008,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93695708,"statfs":{"total":96636764160,"available":95944404992,"internally_reserved":0,"allocated":259080192,"data_stored":258798359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257327104,"data_stored":257290974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192536576,"data_stored":192460244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264785920,"data_stored":264745718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:28.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:28.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'16360", 'reported_seq': 16584, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:28.016520+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:28.016520+0000', 'last_peered': '2024-09-11T01:22:28.016520+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:28.016520+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15884288, 'num_objects': 3878, 'num_object_clones': 0, 'num_object_copies': 7756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16360, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -170}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:28.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'16220", 'reported_seq': 16388, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:25.938978+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:25.938978+0000', 'last_peered': '2024-09-11T01:22:25.938978+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:25.938978+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'6200", 'ondisk_log_start': "39'6200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15667224, 'num_objects': 3826, 'num_object_clones': 0, 'num_object_copies': 7652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3826, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14281, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'16435", 'reported_seq': 16702, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.990868+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:27.990868+0000', 'last_peered': '2024-09-11T01:22:27.990868+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:27.990868+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'6400", 'ondisk_log_start': "39'6400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15450595, 'num_objects': 3773, 'num_object_clones': 0, 'num_object_copies': 7546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3773, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12923, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -276}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16888", 'reported_seq': 17176, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.978815+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:27.978815+0000', 'last_peered': '2024-09-11T01:22:27.978815+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:27.978815+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6800", 'ondisk_log_start': "39'6800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16073670, 'num_objects': 3926, 'num_object_clones': 0, 'num_object_copies': 7852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3926, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13641, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -280}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'16668", 'reported_seq': 16947, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:28.015866+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:28.015866+0000', 'last_peered': '2024-09-11T01:22:28.015866+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:28.015866+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16080896, 'num_objects': 3926, 'num_object_clones': 0, 'num_object_copies': 7852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13032, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -104}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'16238", 'reported_seq': 20640, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:25.929744+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:25.929744+0000', 'last_peered': '2024-09-11T01:22:25.929744+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:25.929744+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'6200", 'ondisk_log_start': "39'6200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15978979, 'num_objects': 3902, 'num_object_clones': 0, 'num_object_copies': 7804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1875, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3902, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14007, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2153, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2027}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:28.913 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'16608", 'reported_seq': 16762, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:25.936478+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:25.936478+0000', 'last_peered': '2024-09-11T01:22:25.936478+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:25.936478+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16195584, 'num_objects': 3954, 'num_object_clones': 0, 'num_object_copies': 7908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14245, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -193}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'16730", 'reported_seq': 17015, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.964161+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:27.964161+0000', 'last_peered': '2024-09-11T01:22:27.964161+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:27.964161+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16039936, 'num_objects': 3916, 'num_object_clones': 0, 'num_object_copies': 7832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13123, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'16694", 'reported_seq': 16994, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.998755+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:27.998755+0000', 'last_peered': '2024-09-11T01:22:27.998755+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:27.998755+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15998976, 'num_objects': 3906, 'num_object_clones': 0, 'num_object_copies': 7812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13229, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:28.914 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'16336", 'reported_seq': 16576, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.933128+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:27.933128+0000', 'last_peered': '2024-09-11T01:22:27.933128+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:27.933128+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15573958, 'num_objects': 3804, 'num_object_clones': 0, 'num_object_copies': 7608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3804, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12993, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -298}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:28.914 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:29.625 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:29.625 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38615/124488 objects degraded (31.019%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":550,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":62244,"data_bytes":254903049,"bytes_used":1857454080,"bytes_avail":288052838400,"bytes_total":289910292480,"degraded_objects":38615,"degraded_total":124488,"degraded_ratio":0.31019054045369837,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":499},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:29.657 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:29.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:30.262 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:30.264 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:30.285 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-11T01:22:28.599583+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":254903049,"num_objects":62244,"num_object_clones":0,"num_object_copies":124488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62244,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":223001,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11020,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1813920,"kb_used_data":701284,"kb_used_omap":12,"kb_used_meta":1112563,"kb_avail":281301600,"statfs":{"total":289910292480,"available":288052838400,"internally_reserved":0,"allocated":718114816,"data_stored":717229120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1139264519},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,7,7,16,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24584192,"num_objects":-6002,"num_object_clones":0,"num_object_copies":-12004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6002,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005053"},"pg_stats":[{"pgid":"2.f","version":"43'16360","reported_seq":16584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.016520+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:28.016520+0000","last_peered":"2024-09-11T01:22:28.016520+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:28.016520+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15884288,"num_objects":3878,"num_object_clones":0,"num_object_copies":7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16528","reported_seq":16607,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.936801+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:25.936801+0000","last_peered":"2024-09-11T01:22:25.936801+0000","last_clean":"2024-09-11T01:22:25.936801+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:25.936801+0000","last_undegraded":"2024-09-11T01:22:25.936801+0000","last_fullsized":"2024-09-11T01:22:25.936801+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16360390,"num_objects":3996,"num_object_clones":0,"num_object_copies":7992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3996,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14626,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16220","reported_seq":16388,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.938978+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:25.938978+0000","last_peered":"2024-09-11T01:22:25.938978+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:25.938978+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15667224,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14281,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-137}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16411","reported_seq":16457,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.938143+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:25.938143+0000","last_peered":"2024-09-11T01:22:25.938143+0000","last_clean":"2024-09-11T01:22:25.938143+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:25.938143+0000","last_undegraded":"2024-09-11T01:22:25.938143+0000","last_fullsized":"2024-09-11T01:22:25.938143+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16411,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16446","reported_seq":16568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:27.991265+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:27.991265+0000","last_peered":"2024-09-11T01:22:27.991265+0000","last_clean":"2024-09-11T01:22:27.991265+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:27.991265+0000","last_undegraded":"2024-09-11T01:22:27.991265+0000","last_fullsized":"2024-09-11T01:22:27.991265+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16435","reported_seq":16702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.990868+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:27.990868+0000","last_peered":"2024-09-11T01:22:27.990868+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:27.990868+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450595,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12923,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-276},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16888","reported_seq":17176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.978815+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:27.978815+0000","last_peered":"2024-09-11T01:22:27.978815+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:27.978815+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16073670,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13641,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-280},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16523","reported_seq":16642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:28.026251+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:28.026251+0000","last_peered":"2024-09-11T01:22:28.026251+0000","last_clean":"2024-09-11T01:22:28.026251+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:28.026251+0000","last_undegraded":"2024-09-11T01:22:28.026251+0000","last_fullsized":"2024-09-11T01:22:28.026251+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16381","reported_seq":24725,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.935925+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:25.935925+0000","last_peered":"2024-09-11T01:22:25.935925+0000","last_clean":"2024-09-11T01:22:25.935925+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:25.935925+0000","last_undegraded":"2024-09-11T01:22:25.935925+0000","last_fullsized":"2024-09-11T01:22:25.935925+0000","mapping_epoch":40,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15843811,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14248,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16668","reported_seq":16947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.015866+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:28.015866+0000","last_peered":"2024-09-11T01:22:28.015866+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:28.015866+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13032,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16238","reported_seq":20640,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.929744+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:25.929744+0000","last_peered":"2024-09-11T01:22:25.929744+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:25.929744+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6200","ondisk_log_start":"39'6200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978979,"num_objects":3902,"num_object_clones":0,"num_object_copies":7804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3902,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14007,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2153,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16608","reported_seq":16762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:25.936478+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:25.936478+0000","last_peered":"2024-09-11T01:22:25.936478+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:25.936478+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16195584,"num_objects":3954,"num_object_clones":0,"num_object_copies":7908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14245,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-193}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16470","reported_seq":24941,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:25.848476+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:25.848476+0000","last_peered":"2024-09-11T01:22:25.848476+0000","last_clean":"2024-09-11T01:22:25.848476+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:25.848476+0000","last_undegraded":"2024-09-11T01:22:25.848476+0000","last_fullsized":"2024-09-11T01:22:25.848476+0000","mapping_epoch":42,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15900672,"num_objects":3882,"num_object_clones":0,"num_object_copies":7764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13160,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16730","reported_seq":17015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.964161+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:27.964161+0000","last_peered":"2024-09-11T01:22:27.964161+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:27.964161+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16039936,"num_objects":3916,"num_object_clones":0,"num_object_copies":7832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13123,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16694","reported_seq":16994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.998755+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:27.998755+0000","last_peered":"2024-09-11T01:22:27.998755+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:27.998755+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16336","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.933128+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:27.933128+0000","last_peered":"2024-09-11T01:22:27.933128+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:27.933128+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15573958,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12993,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":254903030,"num_objects":62242,"num_object_clones":0,"num_object_copies":124484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62242,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":222999,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11019,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":714649600,"data_stored":714496936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566416,"kb_used_data":259832,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93805424,"statfs":{"total":96636764160,"available":96056754176,"internally_reserved":0,"allocated":266067968,"data_stored":265786140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":313913006},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790038,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571372,"kb_used_data":188444,"kb_used_omap":4,"kb_used_meta":382907,"kb_avail":93800468,"statfs":{"total":96636764160,"available":96051679232,"internally_reserved":0,"allocated":192966656,"data_stored":192644621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392097455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,5,7,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676132,"kb_used_data":253008,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93695708,"statfs":{"total":96636764160,"available":95944404992,"internally_reserved":0,"allocated":259080192,"data_stored":258798359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257327104,"data_stored":257290974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":192536576,"data_stored":192460244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264785920,"data_stored":264745718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:33.288 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:33.851 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:33.852 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:33.876 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-11T01:22:32.600827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250708745,"num_objects":61220,"num_object_clones":0,"num_object_copies":122440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61220,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":224025,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11124,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1816068,"kb_used_data":689552,"kb_used_omap":12,"kb_used_meta":1126451,"kb_avail":281299452,"statfs":{"total":289910292480,"available":288050638848,"internally_reserved":0,"allocated":706101248,"data_stored":705215552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1153485831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,6,7,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23359488,"num_objects":-5703,"num_object_clones":0,"num_object_copies":-11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5703,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004736"},"pg_stats":[{"pgid":"2.f","version":"43'16360","reported_seq":16584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.016520+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:28.016520+0000","last_peered":"2024-09-11T01:22:28.016520+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:28.016520+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15884288,"num_objects":3878,"num_object_clones":0,"num_object_copies":7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16688","reported_seq":16768,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.968094+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:30.968094+0000","last_peered":"2024-09-11T01:22:30.968094+0000","last_clean":"2024-09-11T01:22:30.968094+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:30.968094+0000","last_undegraded":"2024-09-11T01:22:30.968094+0000","last_fullsized":"2024-09-11T01:22:30.968094+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15705030,"num_objects":3836,"num_object_clones":0,"num_object_copies":7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3836,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14786,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16371","reported_seq":16540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.842520+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:30.842520+0000","last_peered":"2024-09-11T01:22:30.842520+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:30.842520+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048728,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14432,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-288}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16546","reported_seq":16590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.969604+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:30.969604+0000","last_peered":"2024-09-11T01:22:30.969604+0000","last_clean":"2024-09-11T01:22:30.969604+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:30.969604+0000","last_undegraded":"2024-09-11T01:22:30.969604+0000","last_fullsized":"2024-09-11T01:22:30.969604+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630336,"num_objects":3816,"num_object_clones":0,"num_object_copies":7632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16546,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16446","reported_seq":16568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:27.991265+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:27.991265+0000","last_peered":"2024-09-11T01:22:27.991265+0000","last_clean":"2024-09-11T01:22:27.991265+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:27.991265+0000","last_undegraded":"2024-09-11T01:22:27.991265+0000","last_fullsized":"2024-09-11T01:22:27.991265+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16435","reported_seq":16702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.990868+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:27.990868+0000","last_peered":"2024-09-11T01:22:27.990868+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:27.990868+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450595,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12923,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-276},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16888","reported_seq":17176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.978815+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:27.978815+0000","last_peered":"2024-09-11T01:22:27.978815+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:27.978815+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16073670,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13641,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-280},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16523","reported_seq":16642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:28.026251+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:28.026251+0000","last_peered":"2024-09-11T01:22:28.026251+0000","last_clean":"2024-09-11T01:22:28.026251+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:28.026251+0000","last_undegraded":"2024-09-11T01:22:28.026251+0000","last_fullsized":"2024-09-11T01:22:28.026251+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16527","reported_seq":24871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.971594+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:30.971594+0000","last_peered":"2024-09-11T01:22:30.971594+0000","last_clean":"2024-09-11T01:22:30.971594+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:30.971594+0000","last_undegraded":"2024-09-11T01:22:30.971594+0000","last_fullsized":"2024-09-11T01:22:30.971594+0000","mapping_epoch":40,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15245795,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14394,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16668","reported_seq":16947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.015866+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:28.015866+0000","last_peered":"2024-09-11T01:22:28.015866+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:28.015866+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13032,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16374","reported_seq":20984,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.978619+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:30.978619+0000","last_peered":"2024-09-11T01:22:30.978619+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:30.978619+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15421923,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14143,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2257,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16745","reported_seq":16900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.950323+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:30.950323+0000","last_peered":"2024-09-11T01:22:30.950323+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:30.950323+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15634432,"num_objects":3817,"num_object_clones":0,"num_object_copies":7634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14382,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-330}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16629","reported_seq":25100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.963691+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:30.963691+0000","last_peered":"2024-09-11T01:22:30.963691+0000","last_clean":"2024-09-11T01:22:30.963691+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:30.963691+0000","last_undegraded":"2024-09-11T01:22:30.963691+0000","last_fullsized":"2024-09-11T01:22:30.963691+0000","mapping_epoch":42,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15249408,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16730","reported_seq":17015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.964161+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:27.964161+0000","last_peered":"2024-09-11T01:22:27.964161+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:27.964161+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16039936,"num_objects":3916,"num_object_clones":0,"num_object_copies":7832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13123,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16694","reported_seq":16994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.998755+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:27.998755+0000","last_peered":"2024-09-11T01:22:27.998755+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:27.998755+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16336","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.933128+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:27.933128+0000","last_peered":"2024-09-11T01:22:27.933128+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:27.933128+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15573958,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12993,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250708726,"num_objects":61218,"num_object_clones":0,"num_object_copies":122436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61218,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":224023,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11123,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":702779392,"data_stored":702626728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583932,"kb_used_data":255148,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93787908,"statfs":{"total":96636764160,"available":96038817792,"internally_reserved":0,"allocated":261271552,"data_stored":260989724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790039,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556004,"kb_used_data":181396,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93815836,"statfs":{"total":96636764160,"available":96067416064,"internally_reserved":0,"allocated":185749504,"data_stored":185427469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676132,"kb_used_data":253008,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93695708,"statfs":{"total":96636764160,"available":95944404992,"internally_reserved":0,"allocated":259080192,"data_stored":258798359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257327104,"data_stored":257290974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185364480,"data_stored":185288148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260087808,"data_stored":260047606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:33.878 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:34.489 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:34.494 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:34.524 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-11T01:22:32.600827+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":250708745,"num_objects":61220,"num_object_clones":0,"num_object_copies":122440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61220,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":224025,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11124,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1816068,"kb_used_data":689552,"kb_used_omap":12,"kb_used_meta":1126451,"kb_avail":281299452,"statfs":{"total":289910292480,"available":288050638848,"internally_reserved":0,"allocated":706101248,"data_stored":705215552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1153485831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,6,7,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23359488,"num_objects":-5703,"num_object_clones":0,"num_object_copies":-11406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5703,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004736"},"pg_stats":[{"pgid":"2.f","version":"43'16360","reported_seq":16584,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.016520+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:28.016520+0000","last_peered":"2024-09-11T01:22:28.016520+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:28.016520+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15884288,"num_objects":3878,"num_object_clones":0,"num_object_copies":7756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16688","reported_seq":16768,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.968094+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:30.968094+0000","last_peered":"2024-09-11T01:22:30.968094+0000","last_clean":"2024-09-11T01:22:30.968094+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:30.968094+0000","last_undegraded":"2024-09-11T01:22:30.968094+0000","last_fullsized":"2024-09-11T01:22:30.968094+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15705030,"num_objects":3836,"num_object_clones":0,"num_object_copies":7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3836,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14786,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16371","reported_seq":16540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.842520+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:30.842520+0000","last_peered":"2024-09-11T01:22:30.842520+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:30.842520+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048728,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14432,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-288}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16546","reported_seq":16590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.969604+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:30.969604+0000","last_peered":"2024-09-11T01:22:30.969604+0000","last_clean":"2024-09-11T01:22:30.969604+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:30.969604+0000","last_undegraded":"2024-09-11T01:22:30.969604+0000","last_fullsized":"2024-09-11T01:22:30.969604+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630336,"num_objects":3816,"num_object_clones":0,"num_object_copies":7632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16546,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16446","reported_seq":16568,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:27.991265+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:27.991265+0000","last_peered":"2024-09-11T01:22:27.991265+0000","last_clean":"2024-09-11T01:22:27.991265+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:27.991265+0000","last_undegraded":"2024-09-11T01:22:27.991265+0000","last_fullsized":"2024-09-11T01:22:27.991265+0000","mapping_epoch":31,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15692259,"num_objects":3832,"num_object_clones":0,"num_object_copies":7664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3832,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13244,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16435","reported_seq":16702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.990868+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:27.990868+0000","last_peered":"2024-09-11T01:22:27.990868+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:27.990868+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15450595,"num_objects":3773,"num_object_clones":0,"num_object_copies":7546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12923,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-276},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'16888","reported_seq":17176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.978815+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:27.978815+0000","last_peered":"2024-09-11T01:22:27.978815+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:27.978815+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16073670,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13641,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-280},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16523","reported_seq":16642,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:28.026251+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:28.026251+0000","last_peered":"2024-09-11T01:22:28.026251+0000","last_clean":"2024-09-11T01:22:28.026251+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:28.026251+0000","last_undegraded":"2024-09-11T01:22:28.026251+0000","last_fullsized":"2024-09-11T01:22:28.026251+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15978496,"num_objects":3901,"num_object_clones":0,"num_object_copies":7802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16527","reported_seq":24871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.971594+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:30.971594+0000","last_peered":"2024-09-11T01:22:30.971594+0000","last_clean":"2024-09-11T01:22:30.971594+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:30.971594+0000","last_undegraded":"2024-09-11T01:22:30.971594+0000","last_fullsized":"2024-09-11T01:22:30.971594+0000","mapping_epoch":40,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15245795,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14394,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16668","reported_seq":16947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.015866+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:28.015866+0000","last_peered":"2024-09-11T01:22:28.015866+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:28.015866+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16080896,"num_objects":3926,"num_object_clones":0,"num_object_copies":7852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13032,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-104}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671638+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671638+0000","last_peered":"2024-09-11T01:20:45.671638+0000","last_clean":"2024-09-11T01:20:45.671638+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671638+0000","last_undegraded":"2024-09-11T01:20:45.671638+0000","last_fullsized":"2024-09-11T01:20:45.671638+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16374","reported_seq":20984,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.978619+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:30.978619+0000","last_peered":"2024-09-11T01:22:30.978619+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:30.978619+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15421923,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14143,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2257,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16745","reported_seq":16900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.950323+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:30.950323+0000","last_peered":"2024-09-11T01:22:30.950323+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:30.950323+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15634432,"num_objects":3817,"num_object_clones":0,"num_object_copies":7634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14382,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-330}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16629","reported_seq":25100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.963691+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:30.963691+0000","last_peered":"2024-09-11T01:22:30.963691+0000","last_clean":"2024-09-11T01:22:30.963691+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:30.963691+0000","last_undegraded":"2024-09-11T01:22:30.963691+0000","last_fullsized":"2024-09-11T01:22:30.963691+0000","mapping_epoch":42,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15249408,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16730","reported_seq":17015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.964161+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:27.964161+0000","last_peered":"2024-09-11T01:22:27.964161+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:27.964161+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16039936,"num_objects":3916,"num_object_clones":0,"num_object_copies":7832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13123,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16694","reported_seq":16994,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.998755+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:27.998755+0000","last_peered":"2024-09-11T01:22:27.998755+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:27.998755+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15998976,"num_objects":3906,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16336","reported_seq":16576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:27.933128+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:27.933128+0000","last_peered":"2024-09-11T01:22:27.933128+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:27.933128+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15573958,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12993,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-298},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":250708726,"num_objects":61218,"num_object_clones":0,"num_object_copies":122436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61218,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":224023,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11123,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":702779392,"data_stored":702626728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583932,"kb_used_data":255148,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93787908,"statfs":{"total":96636764160,"available":96038817792,"internally_reserved":0,"allocated":261271552,"data_stored":260989724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790039,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556004,"kb_used_data":181396,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93815836,"statfs":{"total":96636764160,"available":96067416064,"internally_reserved":0,"allocated":185749504,"data_stored":185427469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676132,"kb_used_data":253008,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93695708,"statfs":{"total":96636764160,"available":95944404992,"internally_reserved":0,"allocated":259080192,"data_stored":258798359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":433254058},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257327104,"data_stored":257290974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185364480,"data_stored":185288148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260087808,"data_stored":260047606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:34.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'16360", 'reported_seq': 16584, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:28.016520+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:28.016520+0000', 'last_peered': '2024-09-11T01:22:28.016520+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:28.016520+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15884288, 'num_objects': 3878, 'num_object_clones': 0, 'num_object_copies': 7756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3878, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16360, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -170}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'16371", 'reported_seq': 16540, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:30.842520+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:30.842520+0000', 'last_peered': '2024-09-11T01:22:30.842520+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:30.842520+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15048728, 'num_objects': 3675, 'num_object_clones': 0, 'num_object_copies': 7350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3675, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14432, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -288}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'16435", 'reported_seq': 16702, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.990868+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:27.990868+0000', 'last_peered': '2024-09-11T01:22:27.990868+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:27.990868+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'6400", 'ondisk_log_start': "39'6400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15450595, 'num_objects': 3773, 'num_object_clones': 0, 'num_object_copies': 7546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3773, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12923, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -276}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'16888", 'reported_seq': 17176, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.978815+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:27.978815+0000', 'last_peered': '2024-09-11T01:22:27.978815+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:27.978815+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'6800", 'ondisk_log_start': "39'6800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16073670, 'num_objects': 3926, 'num_object_clones': 0, 'num_object_copies': 7852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3926, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13641, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -280}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:34.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'16668", 'reported_seq': 16947, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:28.015866+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:28.015866+0000', 'last_peered': '2024-09-11T01:22:28.015866+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:28.015866+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16080896, 'num_objects': 3926, 'num_object_clones': 0, 'num_object_copies': 7852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13032, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -104}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'16374", 'reported_seq': 20984, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:30.978619+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:30.978619+0000', 'last_peered': '2024-09-11T01:22:30.978619+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:30.978619+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15421923, 'num_objects': 3766, 'num_object_clones': 0, 'num_object_copies': 7532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1771, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3766, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14143, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2257, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1995}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'16745", 'reported_seq': 16900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:30.950323+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:30.950323+0000', 'last_peered': '2024-09-11T01:22:30.950323+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:30.950323+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15634432, 'num_objects': 3817, 'num_object_clones': 0, 'num_object_copies': 7634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14382, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -330}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'16730", 'reported_seq': 17015, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.964161+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:27.964161+0000', 'last_peered': '2024-09-11T01:22:27.964161+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:27.964161+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16039936, 'num_objects': 3916, 'num_object_clones': 0, 'num_object_copies': 7832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13123, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'16694", 'reported_seq': 16994, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.998755+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:27.998755+0000', 'last_peered': '2024-09-11T01:22:27.998755+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:27.998755+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15998976, 'num_objects': 3906, 'num_object_clones': 0, 'num_object_copies': 7812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13229, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:34.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'16336", 'reported_seq': 16576, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:27.933128+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:27.933128+0000', 'last_peered': '2024-09-11T01:22:27.933128+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:27.933128+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'6300", 'ondisk_log_start': "39'6300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15573958, 'num_objects': 3804, 'num_object_clones': 0, 'num_object_copies': 7608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3804, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12993, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -298}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:34.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:35.253 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:35.254 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38511/119782 objects degraded (32.151%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":555,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59891,"data_bytes":245265161,"bytes_used":1839562752,"bytes_avail":288070729728,"bytes_total":289910292480,"degraded_objects":38511,"degraded_total":119782,"degraded_ratio":0.321509074819255,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":498},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:35.285 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:35.285 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:35.897 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:35.898 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:35.921 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T01:22:34.601528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":245265161,"num_objects":59891,"num_object_clones":0,"num_object_copies":119782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59891,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":225354,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11124,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1796448,"kb_used_data":682416,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281319072,"statfs":{"total":289910292480,"available":288070729728,"internally_reserved":0,"allocated":698793984,"data_stored":697908288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1140706311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,8,7,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24498176,"num_objects":-5981,"num_object_clones":0,"num_object_copies":-11962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-198,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5981,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004545"},"pg_stats":[{"pgid":"2.f","version":"43'16507","reported_seq":16731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.053186+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:33.053186+0000","last_peered":"2024-09-11T01:22:33.053186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:33.053186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15282176,"num_objects":3731,"num_object_clones":0,"num_object_copies":7462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16507,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16688","reported_seq":16768,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.968094+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:30.968094+0000","last_peered":"2024-09-11T01:22:30.968094+0000","last_clean":"2024-09-11T01:22:30.968094+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:30.968094+0000","last_undegraded":"2024-09-11T01:22:30.968094+0000","last_fullsized":"2024-09-11T01:22:30.968094+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15705030,"num_objects":3836,"num_object_clones":0,"num_object_copies":7672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3836,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14786,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16371","reported_seq":16540,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.842520+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:30.842520+0000","last_peered":"2024-09-11T01:22:30.842520+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:30.842520+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15048728,"num_objects":3675,"num_object_clones":0,"num_object_copies":7350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3675,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14432,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-288}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16546","reported_seq":16590,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.969604+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:30.969604+0000","last_peered":"2024-09-11T01:22:30.969604+0000","last_clean":"2024-09-11T01:22:30.969604+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:30.969604+0000","last_undegraded":"2024-09-11T01:22:30.969604+0000","last_fullsized":"2024-09-11T01:22:30.969604+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15630336,"num_objects":3816,"num_object_clones":0,"num_object_copies":7632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16546,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16578","reported_seq":16700,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:33.066269+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:33.066269+0000","last_peered":"2024-09-11T01:22:33.066269+0000","last_clean":"2024-09-11T01:22:33.066269+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:33.066269+0000","last_undegraded":"2024-09-11T01:22:33.066269+0000","last_fullsized":"2024-09-11T01:22:33.066269+0000","mapping_epoch":31,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15151587,"num_objects":3700,"num_object_clones":0,"num_object_copies":7400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13376,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16581","reported_seq":16846,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.076713+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:33.076713+0000","last_peered":"2024-09-11T01:22:33.076713+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:33.076713+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852579,"num_objects":3627,"num_object_clones":0,"num_object_copies":7254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3627,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13069,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-422},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17032","reported_seq":17320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.037187+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:33.037187+0000","last_peered":"2024-09-11T01:22:33.037187+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:33.037187+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15483846,"num_objects":3782,"num_object_clones":0,"num_object_copies":7564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3782,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13785,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-424},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16676","reported_seq":16796,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:33.073913+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:33.073913+0000","last_peered":"2024-09-11T01:22:33.073913+0000","last_clean":"2024-09-11T01:22:33.073913+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:33.073913+0000","last_undegraded":"2024-09-11T01:22:33.073913+0000","last_fullsized":"2024-09-11T01:22:33.073913+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15351808,"num_objects":3748,"num_object_clones":0,"num_object_copies":7496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13629,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16527","reported_seq":24871,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.971594+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:30.971594+0000","last_peered":"2024-09-11T01:22:30.971594+0000","last_clean":"2024-09-11T01:22:30.971594+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:30.971594+0000","last_undegraded":"2024-09-11T01:22:30.971594+0000","last_fullsized":"2024-09-11T01:22:30.971594+0000","mapping_epoch":40,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15245795,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14394,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16815","reported_seq":17095,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.006902+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:33.006902+0000","last_peered":"2024-09-11T01:22:33.006902+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:33.006902+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15478784,"num_objects":3779,"num_object_clones":0,"num_object_copies":7558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3779,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13179,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16374","reported_seq":20984,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.978619+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:30.978619+0000","last_peered":"2024-09-11T01:22:30.978619+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:30.978619+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6300","ondisk_log_start":"39'6300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15421923,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14143,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2257,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1995}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16745","reported_seq":16900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:30.950323+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:30.950323+0000","last_peered":"2024-09-11T01:22:30.950323+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:30.950323+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15634432,"num_objects":3817,"num_object_clones":0,"num_object_copies":7634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14382,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-330}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16629","reported_seq":25100,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:30.963691+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:30.963691+0000","last_peered":"2024-09-11T01:22:30.963691+0000","last_clean":"2024-09-11T01:22:30.963691+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:30.963691+0000","last_undegraded":"2024-09-11T01:22:30.963691+0000","last_fullsized":"2024-09-11T01:22:30.963691+0000","mapping_epoch":42,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15249408,"num_objects":3723,"num_object_clones":0,"num_object_copies":7446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13319,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'16892","reported_seq":17177,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.043854+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:33.043854+0000","last_peered":"2024-09-11T01:22:33.043854+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:33.043854+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15376384,"num_objects":3754,"num_object_clones":0,"num_object_copies":7508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13285,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16845","reported_seq":17145,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:32.837566+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:32.837566+0000","last_peered":"2024-09-11T01:22:32.837566+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:32.837566+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15380480,"num_objects":3755,"num_object_clones":0,"num_object_copies":7510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13380,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-310}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16483","reported_seq":16723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.087209+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:33.087209+0000","last_peered":"2024-09-11T01:22:33.087209+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:33.087209+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6400","ondisk_log_start":"39'6400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14971846,"num_objects":3657,"num_object_clones":0,"num_object_copies":7314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13140,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-445},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":245265142,"num_objects":59889,"num_object_clones":0,"num_object_copies":119778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38511,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59889,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":225352,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11123,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":695504896,"data_stored":695352232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583932,"kb_used_data":255148,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93787908,"statfs":{"total":96636764160,"available":96038817792,"internally_reserved":0,"allocated":261271552,"data_stored":260989724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790039,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556004,"kb_used_data":181396,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93815836,"statfs":{"total":96636764160,"available":96067416064,"internally_reserved":0,"allocated":185749504,"data_stored":185427469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656512,"kb_used_data":245872,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93715328,"statfs":{"total":96636764160,"available":95964495872,"internally_reserved":0,"allocated":251772928,"data_stored":251491095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250052608,"data_stored":250016478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":185364480,"data_stored":185288148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260087808,"data_stored":260047606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:38.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:39.507 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:39.507 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:39.534 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-11T01:22:38.606522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235930377,"num_objects":57612,"num_object_clones":0,"num_object_copies":115224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57612,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":227633,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11230,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1774292,"kb_used_data":664408,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281341228,"statfs":{"total":289910292480,"available":288093417472,"internally_reserved":0,"allocated":680353792,"data_stored":679468096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1136446471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,9,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24358912,"num_objects":-5947,"num_object_clones":0,"num_object_copies":-11894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5947,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007846"},"pg_stats":[{"pgid":"2.f","version":"43'16646","reported_seq":16870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.102494+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:38.102494+0000","last_peered":"2024-09-11T01:22:38.102494+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:38.102494+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14712832,"num_objects":3592,"num_object_clones":0,"num_object_copies":7184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16646,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16829","reported_seq":16910,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.970813+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:35.970813+0000","last_peered":"2024-09-11T01:22:35.970813+0000","last_clean":"2024-09-11T01:22:35.970813+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:35.970813+0000","last_undegraded":"2024-09-11T01:22:35.970813+0000","last_fullsized":"2024-09-11T01:22:35.970813+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127494,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14927,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16503","reported_seq":16672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.930687+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:35.930687+0000","last_peered":"2024-09-11T01:22:35.930687+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:35.930687+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14508056,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14564,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16692","reported_seq":16738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.985659+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:35.985659+0000","last_peered":"2024-09-11T01:22:35.985659+0000","last_clean":"2024-09-11T01:22:35.985659+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:35.985659+0000","last_undegraded":"2024-09-11T01:22:35.985659+0000","last_fullsized":"2024-09-11T01:22:35.985659+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032320,"num_objects":3670,"num_object_clones":0,"num_object_copies":7340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16735","reported_seq":16857,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:38.158391+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:38.158391+0000","last_peered":"2024-09-11T01:22:38.158391+0000","last_clean":"2024-09-11T01:22:38.158391+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:38.158391+0000","last_undegraded":"2024-09-11T01:22:38.158391+0000","last_fullsized":"2024-09-11T01:22:38.158391+0000","mapping_epoch":31,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14508515,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13533,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16733","reported_seq":16999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.120498+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:38.120498+0000","last_peered":"2024-09-11T01:22:38.120498+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:38.120498+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14229987,"num_objects":3475,"num_object_clones":0,"num_object_copies":6950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-574},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17178","reported_seq":17464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.170267+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:38.170267+0000","last_peered":"2024-09-11T01:22:38.170267+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:38.170267+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14885830,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13931,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-570},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16816","reported_seq":16935,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:38.162938+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:38.162938+0000","last_peered":"2024-09-11T01:22:38.162938+0000","last_clean":"2024-09-11T01:22:38.162938+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:38.162938+0000","last_undegraded":"2024-09-11T01:22:38.162938+0000","last_fullsized":"2024-09-11T01:22:38.162938+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14778368,"num_objects":3608,"num_object_clones":0,"num_object_copies":7216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13769,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16666","reported_seq":25010,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.921381+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:35.921381+0000","last_peered":"2024-09-11T01:22:35.921381+0000","last_clean":"2024-09-11T01:22:35.921381+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:35.921381+0000","last_undegraded":"2024-09-11T01:22:35.921381+0000","last_fullsized":"2024-09-11T01:22:35.921381+0000","mapping_epoch":40,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14676451,"num_objects":3584,"num_object_clones":0,"num_object_copies":7168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14533,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16944","reported_seq":17224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.119291+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:38.119291+0000","last_peered":"2024-09-11T01:22:38.119291+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:38.119291+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13308,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16501","reported_seq":21323,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.950314+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:35.950314+0000","last_peered":"2024-09-11T01:22:35.950314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:35.950314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14901731,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14270,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2363,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1974}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16897","reported_seq":17052,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.986106+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:35.986106+0000","last_peered":"2024-09-11T01:22:35.986106+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:35.986106+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15011840,"num_objects":3665,"num_object_clones":0,"num_object_copies":7330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16758","reported_seq":25229,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.036590+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:36.036590+0000","last_peered":"2024-09-11T01:22:36.036590+0000","last_clean":"2024-09-11T01:22:36.036590+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:36.036590+0000","last_undegraded":"2024-09-11T01:22:36.036590+0000","last_fullsized":"2024-09-11T01:22:36.036590+0000","mapping_epoch":42,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14721024,"num_objects":3594,"num_object_clones":0,"num_object_copies":7188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17061","reported_seq":17344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.143857+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:38.143857+0000","last_peered":"2024-09-11T01:22:38.143857+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:38.143857+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14684160,"num_objects":3585,"num_object_clones":0,"num_object_copies":7170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13454,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16986","reported_seq":17286,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.137720+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:38.137720+0000","last_peered":"2024-09-11T01:22:38.137720+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:38.137720+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14802944,"num_objects":3614,"num_object_clones":0,"num_object_copies":7228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13521,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16623","reported_seq":16863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.140892+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:38.140892+0000","last_peered":"2024-09-11T01:22:38.140892+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:38.140892+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14398406,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13280,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-585},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":235930358,"num_objects":57610,"num_object_clones":0,"num_object_copies":115220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57610,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":227631,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11229,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676835328,"data_stored":676682664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566888,"kb_used_data":250584,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804952,"statfs":{"total":96636764160,"available":96056270848,"internally_reserved":0,"allocated":256598016,"data_stored":256316188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790040,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553496,"kb_used_data":174728,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93818344,"statfs":{"total":96636764160,"available":96069984256,"internally_reserved":0,"allocated":178921472,"data_stored":178599437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653908,"kb_used_data":239096,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93717932,"statfs":{"total":96636764160,"available":95967162368,"internally_reserved":0,"allocated":244834304,"data_stored":244552471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242999296,"data_stored":242963166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178421760,"data_stored":178345428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255414272,"data_stored":255374070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:39.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:40.095 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:40.098 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:40.126 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-11T01:22:38.606522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235930377,"num_objects":57612,"num_object_clones":0,"num_object_copies":115224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57612,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":227633,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11230,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1774292,"kb_used_data":664408,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281341228,"statfs":{"total":289910292480,"available":288093417472,"internally_reserved":0,"allocated":680353792,"data_stored":679468096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1136446471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,9,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24358912,"num_objects":-5947,"num_object_clones":0,"num_object_copies":-11894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5947,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007846"},"pg_stats":[{"pgid":"2.f","version":"43'16646","reported_seq":16870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.102494+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:38.102494+0000","last_peered":"2024-09-11T01:22:38.102494+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:38.102494+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14712832,"num_objects":3592,"num_object_clones":0,"num_object_copies":7184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16646,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16829","reported_seq":16910,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.970813+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:35.970813+0000","last_peered":"2024-09-11T01:22:35.970813+0000","last_clean":"2024-09-11T01:22:35.970813+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:35.970813+0000","last_undegraded":"2024-09-11T01:22:35.970813+0000","last_fullsized":"2024-09-11T01:22:35.970813+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127494,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14927,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16503","reported_seq":16672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.930687+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:35.930687+0000","last_peered":"2024-09-11T01:22:35.930687+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:35.930687+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14508056,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14564,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16692","reported_seq":16738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.985659+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:35.985659+0000","last_peered":"2024-09-11T01:22:35.985659+0000","last_clean":"2024-09-11T01:22:35.985659+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:35.985659+0000","last_undegraded":"2024-09-11T01:22:35.985659+0000","last_fullsized":"2024-09-11T01:22:35.985659+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032320,"num_objects":3670,"num_object_clones":0,"num_object_copies":7340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16735","reported_seq":16857,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:38.158391+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:38.158391+0000","last_peered":"2024-09-11T01:22:38.158391+0000","last_clean":"2024-09-11T01:22:38.158391+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:38.158391+0000","last_undegraded":"2024-09-11T01:22:38.158391+0000","last_fullsized":"2024-09-11T01:22:38.158391+0000","mapping_epoch":31,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14508515,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13533,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16733","reported_seq":16999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.120498+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:38.120498+0000","last_peered":"2024-09-11T01:22:38.120498+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:38.120498+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14229987,"num_objects":3475,"num_object_clones":0,"num_object_copies":6950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-574},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17178","reported_seq":17464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.170267+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:38.170267+0000","last_peered":"2024-09-11T01:22:38.170267+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:38.170267+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14885830,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13931,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-570},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16816","reported_seq":16935,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:38.162938+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:38.162938+0000","last_peered":"2024-09-11T01:22:38.162938+0000","last_clean":"2024-09-11T01:22:38.162938+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:38.162938+0000","last_undegraded":"2024-09-11T01:22:38.162938+0000","last_fullsized":"2024-09-11T01:22:38.162938+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14778368,"num_objects":3608,"num_object_clones":0,"num_object_copies":7216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13769,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16666","reported_seq":25010,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.921381+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:35.921381+0000","last_peered":"2024-09-11T01:22:35.921381+0000","last_clean":"2024-09-11T01:22:35.921381+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:35.921381+0000","last_undegraded":"2024-09-11T01:22:35.921381+0000","last_fullsized":"2024-09-11T01:22:35.921381+0000","mapping_epoch":40,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14676451,"num_objects":3584,"num_object_clones":0,"num_object_copies":7168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14533,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16944","reported_seq":17224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.119291+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:38.119291+0000","last_peered":"2024-09-11T01:22:38.119291+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:38.119291+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13308,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16501","reported_seq":21323,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.950314+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:35.950314+0000","last_peered":"2024-09-11T01:22:35.950314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:35.950314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14901731,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14270,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2363,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1974}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16897","reported_seq":17052,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.986106+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:35.986106+0000","last_peered":"2024-09-11T01:22:35.986106+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:35.986106+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15011840,"num_objects":3665,"num_object_clones":0,"num_object_copies":7330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16758","reported_seq":25229,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.036590+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:36.036590+0000","last_peered":"2024-09-11T01:22:36.036590+0000","last_clean":"2024-09-11T01:22:36.036590+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:36.036590+0000","last_undegraded":"2024-09-11T01:22:36.036590+0000","last_fullsized":"2024-09-11T01:22:36.036590+0000","mapping_epoch":42,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14721024,"num_objects":3594,"num_object_clones":0,"num_object_copies":7188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17061","reported_seq":17344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.143857+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:38.143857+0000","last_peered":"2024-09-11T01:22:38.143857+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:38.143857+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14684160,"num_objects":3585,"num_object_clones":0,"num_object_copies":7170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13454,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16986","reported_seq":17286,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.137720+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:38.137720+0000","last_peered":"2024-09-11T01:22:38.137720+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:38.137720+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14802944,"num_objects":3614,"num_object_clones":0,"num_object_copies":7228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13521,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16623","reported_seq":16863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.140892+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:38.140892+0000","last_peered":"2024-09-11T01:22:38.140892+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:38.140892+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14398406,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13280,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-585},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":235930358,"num_objects":57610,"num_object_clones":0,"num_object_copies":115220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57610,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":227631,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11229,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676835328,"data_stored":676682664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566888,"kb_used_data":250584,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804952,"statfs":{"total":96636764160,"available":96056270848,"internally_reserved":0,"allocated":256598016,"data_stored":256316188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790040,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553496,"kb_used_data":174728,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93818344,"statfs":{"total":96636764160,"available":96069984256,"internally_reserved":0,"allocated":178921472,"data_stored":178599437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653908,"kb_used_data":239096,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93717932,"statfs":{"total":96636764160,"available":95967162368,"internally_reserved":0,"allocated":244834304,"data_stored":244552471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242999296,"data_stored":242963166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178421760,"data_stored":178345428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255414272,"data_stored":255374070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:40.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:40.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'16646", 'reported_seq': 16870, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.102494+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:38.102494+0000', 'last_peered': '2024-09-11T01:22:38.102494+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:38.102494+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14712832, 'num_objects': 3592, 'num_object_clones': 0, 'num_object_copies': 7184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16646, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -456}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:40.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'16503", 'reported_seq': 16672, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:35.930687+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:35.930687+0000', 'last_peered': '2024-09-11T01:22:35.930687+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:35.930687+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'6500", 'ondisk_log_start': "39'6500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14508056, 'num_objects': 3543, 'num_object_clones': 0, 'num_object_copies': 7086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3543, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14564, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -420}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'16733", 'reported_seq': 16999, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.120498+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:38.120498+0000', 'last_peered': '2024-09-11T01:22:38.120498+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:38.120498+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14229987, 'num_objects': 3475, 'num_object_clones': 0, 'num_object_copies': 6950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3475, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13221, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -574}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'17178", 'reported_seq': 17464, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.170267+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:38.170267+0000', 'last_peered': '2024-09-11T01:22:38.170267+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:38.170267+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'7100", 'ondisk_log_start': "39'7100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14885830, 'num_objects': 3636, 'num_object_clones': 0, 'num_object_copies': 7272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3636, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 13931, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -570}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'16944", 'reported_seq': 17224, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.119291+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:38.119291+0000', 'last_peered': '2024-09-11T01:22:38.119291+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:38.119291+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'6900", 'ondisk_log_start': "39'6900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14950400, 'num_objects': 3650, 'num_object_clones': 0, 'num_object_copies': 7300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13308, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -380}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'16501", 'reported_seq': 21323, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:35.950314+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:35.950314+0000', 'last_peered': '2024-09-11T01:22:35.950314+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:35.950314+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'6500", 'ondisk_log_start': "39'6500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14901731, 'num_objects': 3639, 'num_object_clones': 0, 'num_object_copies': 7278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1665, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3639, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14270, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2363, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1974}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'16897", 'reported_seq': 17052, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:35.986106+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:35.986106+0000', 'last_peered': '2024-09-11T01:22:35.986106+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:35.986106+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'6800", 'ondisk_log_start': "39'6800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15011840, 'num_objects': 3665, 'num_object_clones': 0, 'num_object_copies': 7330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14534, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -482}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:40.128 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17061", 'reported_seq': 17344, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.143857+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:38.143857+0000', 'last_peered': '2024-09-11T01:22:38.143857+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:38.143857+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14684160, 'num_objects': 3585, 'num_object_clones': 0, 'num_object_copies': 7170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13454, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -532}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.129 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:40.129 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'16986", 'reported_seq': 17286, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.137720+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:38.137720+0000', 'last_peered': '2024-09-11T01:22:38.137720+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:38.137720+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'6900", 'ondisk_log_start': "39'6900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14802944, 'num_objects': 3614, 'num_object_clones': 0, 'num_object_copies': 7228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13521, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.129 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:40.129 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'16623", 'reported_seq': 16863, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:38.140892+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:38.140892+0000', 'last_peered': '2024-09-11T01:22:38.140892+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:38.140892+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14398406, 'num_objects': 3517, 'num_object_clones': 0, 'num_object_copies': 7034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3517, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13280, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -585}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:40.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:40.865 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:40.865 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38405/115224 objects degraded (33.331%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":561,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":57612,"data_bytes":235930377,"bytes_used":1816875008,"bytes_avail":288093417472,"bytes_total":289910292480,"degraded_objects":38405,"degraded_total":115224,"degraded_ratio":0.33330729709088386,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":385},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:40.887 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:40.887 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:41.461 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:41.462 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:41.488 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T01:22:40.607315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":235930377,"num_objects":57612,"num_object_clones":0,"num_object_copies":115224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57612,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":227633,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11230,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1774292,"kb_used_data":664408,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281341228,"statfs":{"total":289910292480,"available":288093417472,"internally_reserved":0,"allocated":680353792,"data_stored":679468096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1136446471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,9,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18972672,"num_objects":-4632,"num_object_clones":0,"num_object_copies":-9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-210,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4632,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007731"},"pg_stats":[{"pgid":"2.f","version":"43'16646","reported_seq":16870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.102494+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:38.102494+0000","last_peered":"2024-09-11T01:22:38.102494+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:38.102494+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14712832,"num_objects":3592,"num_object_clones":0,"num_object_copies":7184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16646,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16829","reported_seq":16910,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.970813+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:35.970813+0000","last_peered":"2024-09-11T01:22:35.970813+0000","last_clean":"2024-09-11T01:22:35.970813+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:35.970813+0000","last_undegraded":"2024-09-11T01:22:35.970813+0000","last_fullsized":"2024-09-11T01:22:35.970813+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15127494,"num_objects":3695,"num_object_clones":0,"num_object_copies":7390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3695,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14927,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16503","reported_seq":16672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.930687+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:35.930687+0000","last_peered":"2024-09-11T01:22:35.930687+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:35.930687+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14508056,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14564,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16692","reported_seq":16738,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.985659+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:35.985659+0000","last_peered":"2024-09-11T01:22:35.985659+0000","last_clean":"2024-09-11T01:22:35.985659+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:35.985659+0000","last_undegraded":"2024-09-11T01:22:35.985659+0000","last_fullsized":"2024-09-11T01:22:35.985659+0000","mapping_epoch":31,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15032320,"num_objects":3670,"num_object_clones":0,"num_object_copies":7340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16692,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16735","reported_seq":16857,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:38.158391+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:38.158391+0000","last_peered":"2024-09-11T01:22:38.158391+0000","last_clean":"2024-09-11T01:22:38.158391+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:38.158391+0000","last_undegraded":"2024-09-11T01:22:38.158391+0000","last_fullsized":"2024-09-11T01:22:38.158391+0000","mapping_epoch":31,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14508515,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13533,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16733","reported_seq":16999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.120498+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:38.120498+0000","last_peered":"2024-09-11T01:22:38.120498+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:38.120498+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14229987,"num_objects":3475,"num_object_clones":0,"num_object_copies":6950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13221,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-574},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17178","reported_seq":17464,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.170267+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:38.170267+0000","last_peered":"2024-09-11T01:22:38.170267+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:38.170267+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14885830,"num_objects":3636,"num_object_clones":0,"num_object_copies":7272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13931,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-570},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16816","reported_seq":16935,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:38.162938+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:38.162938+0000","last_peered":"2024-09-11T01:22:38.162938+0000","last_clean":"2024-09-11T01:22:38.162938+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:38.162938+0000","last_undegraded":"2024-09-11T01:22:38.162938+0000","last_fullsized":"2024-09-11T01:22:38.162938+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14778368,"num_objects":3608,"num_object_clones":0,"num_object_copies":7216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3608,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13769,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16666","reported_seq":25010,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:35.921381+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:35.921381+0000","last_peered":"2024-09-11T01:22:35.921381+0000","last_clean":"2024-09-11T01:22:35.921381+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:35.921381+0000","last_undegraded":"2024-09-11T01:22:35.921381+0000","last_fullsized":"2024-09-11T01:22:35.921381+0000","mapping_epoch":40,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14676451,"num_objects":3584,"num_object_clones":0,"num_object_copies":7168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3584,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14533,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'16944","reported_seq":17224,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.119291+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:38.119291+0000","last_peered":"2024-09-11T01:22:38.119291+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:38.119291+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13308,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16501","reported_seq":21323,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.950314+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:35.950314+0000","last_peered":"2024-09-11T01:22:35.950314+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:35.950314+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6500","ondisk_log_start":"39'6500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14901731,"num_objects":3639,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14270,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2363,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1974}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'16897","reported_seq":17052,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.986106+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:35.986106+0000","last_peered":"2024-09-11T01:22:35.986106+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:35.986106+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15011840,"num_objects":3665,"num_object_clones":0,"num_object_copies":7330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-482}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16758","reported_seq":25229,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.036590+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:36.036590+0000","last_peered":"2024-09-11T01:22:36.036590+0000","last_clean":"2024-09-11T01:22:36.036590+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:36.036590+0000","last_undegraded":"2024-09-11T01:22:36.036590+0000","last_fullsized":"2024-09-11T01:22:36.036590+0000","mapping_epoch":42,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14721024,"num_objects":3594,"num_object_clones":0,"num_object_copies":7188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13448,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17061","reported_seq":17344,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.143857+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:38.143857+0000","last_peered":"2024-09-11T01:22:38.143857+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:38.143857+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14684160,"num_objects":3585,"num_object_clones":0,"num_object_copies":7170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13454,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-532}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'16986","reported_seq":17286,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.137720+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:38.137720+0000","last_peered":"2024-09-11T01:22:38.137720+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:38.137720+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14802944,"num_objects":3614,"num_object_clones":0,"num_object_copies":7228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13521,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16623","reported_seq":16863,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:38.140892+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:38.140892+0000","last_peered":"2024-09-11T01:22:38.140892+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:38.140892+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14398406,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13280,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-585},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":235930358,"num_objects":57610,"num_object_clones":0,"num_object_copies":115220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":57610,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":227631,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11229,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":676835328,"data_stored":676682664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566888,"kb_used_data":250584,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93804952,"statfs":{"total":96636764160,"available":96056270848,"internally_reserved":0,"allocated":256598016,"data_stored":256316188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790040,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553496,"kb_used_data":174728,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93818344,"statfs":{"total":96636764160,"available":96069984256,"internally_reserved":0,"allocated":178921472,"data_stored":178599437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":653908,"kb_used_data":239096,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93717932,"statfs":{"total":96636764160,"available":95967162368,"internally_reserved":0,"allocated":244834304,"data_stored":244552471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":242999296,"data_stored":242963166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":178421760,"data_stored":178345428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":255414272,"data_stored":255374070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:44.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:45.057 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:45.059 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:45.088 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-11T01:22:44.609098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":226288393,"num_objects":55258,"num_object_clones":0,"num_object_copies":110516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55258,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":229987,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11336,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1759672,"kb_used_data":645640,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281355848,"statfs":{"total":289910292480,"available":288108388352,"internally_reserved":0,"allocated":661135360,"data_stored":660249664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1140706311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,9,4,4,5,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24420352,"num_objects":-5962,"num_object_clones":0,"num_object_copies":-11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008270"},"pg_stats":[{"pgid":"2.f","version":"43'16785","reported_seq":17009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.195015+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:43.195015+0000","last_peered":"2024-09-11T01:22:43.195015+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:43.195015+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16977","reported_seq":17058,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:40.999714+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:40.999714+0000","last_peered":"2024-09-11T01:22:40.999714+0000","last_clean":"2024-09-11T01:22:40.999714+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:40.999714+0000","last_undegraded":"2024-09-11T01:22:40.999714+0000","last_fullsized":"2024-09-11T01:22:40.999714+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14521286,"num_objects":3547,"num_object_clones":0,"num_object_copies":7094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3547,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15075,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16644","reported_seq":16813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.859193+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:40.859193+0000","last_peered":"2024-09-11T01:22:40.859193+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:40.859193+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930520,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14705,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-561}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16845","reported_seq":16891,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:41.007550+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:41.007550+0000","last_peered":"2024-09-11T01:22:41.007550+0000","last_clean":"2024-09-11T01:22:41.007550+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:41.007550+0000","last_undegraded":"2024-09-11T01:22:41.007550+0000","last_fullsized":"2024-09-11T01:22:41.007550+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16845,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16891","reported_seq":17011,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:43.219407+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:43.219407+0000","last_peered":"2024-09-11T01:22:43.219407+0000","last_clean":"2024-09-11T01:22:43.219407+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:43.219407+0000","last_undegraded":"2024-09-11T01:22:43.219407+0000","last_fullsized":"2024-09-11T01:22:43.219407+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869539,"num_objects":3387,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3387,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13689,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16886","reported_seq":17151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.133181+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:43.133181+0000","last_peered":"2024-09-11T01:22:43.133181+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:43.133181+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13603299,"num_objects":3322,"num_object_clones":0,"num_object_copies":6644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13374,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-727},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17319","reported_seq":17607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.188612+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:43.188612+0000","last_peered":"2024-09-11T01:22:43.188612+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:43.188612+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14308294,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14072,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-711},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16974","reported_seq":17094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:43.141230+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:43.141230+0000","last_peered":"2024-09-11T01:22:43.141230+0000","last_clean":"2024-09-11T01:22:43.141230+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:43.141230+0000","last_undegraded":"2024-09-11T01:22:43.141230+0000","last_fullsized":"2024-09-11T01:22:43.141230+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14131200,"num_objects":3450,"num_object_clones":0,"num_object_copies":6900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13927,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16812","reported_seq":25156,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:40.987903+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:40.987903+0000","last_peered":"2024-09-11T01:22:40.987903+0000","last_clean":"2024-09-11T01:22:40.987903+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:40.987903+0000","last_undegraded":"2024-09-11T01:22:40.987903+0000","last_fullsized":"2024-09-11T01:22:40.987903+0000","mapping_epoch":40,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14078435,"num_objects":3438,"num_object_clones":0,"num_object_copies":6876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14679,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17090","reported_seq":17370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.174282+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:43.174282+0000","last_peered":"2024-09-11T01:22:43.174282+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:43.174282+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":3504,"num_object_clones":0,"num_object_copies":7008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13454,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16639","reported_seq":21673,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:41.013739+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:41.013739+0000","last_peered":"2024-09-11T01:22:41.013739+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:41.013739+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336483,"num_objects":3501,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1559,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14408,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1942}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17043","reported_seq":17198,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.937720+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:40.937720+0000","last_peered":"2024-09-11T01:22:40.937720+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:40.937720+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14413824,"num_objects":3519,"num_object_clones":0,"num_object_copies":7038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-628}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16885","reported_seq":25355,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:41.090896+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:41.090896+0000","last_peered":"2024-09-11T01:22:41.090896+0000","last_clean":"2024-09-11T01:22:41.090896+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:41.090896+0000","last_undegraded":"2024-09-11T01:22:41.090896+0000","last_fullsized":"2024-09-11T01:22:41.090896+0000","mapping_epoch":42,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13575,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17219","reported_seq":17504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.175858+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:43.175858+0000","last_peered":"2024-09-11T01:22:43.175858+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:43.175858+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17134","reported_seq":17431,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.253706+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:43.253706+0000","last_peered":"2024-09-11T01:22:43.253706+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:43.253706+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14196736,"num_objects":3466,"num_object_clones":0,"num_object_copies":6932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13669,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16779","reported_seq":17019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.236855+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:43.236855+0000","last_peered":"2024-09-11T01:22:43.236855+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:43.236855+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13759430,"num_objects":3361,"num_object_clones":0,"num_object_copies":6722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13436,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-741},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":226288374,"num_objects":55256,"num_object_clones":0,"num_object_copies":110512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55256,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":229985,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11335,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":657522688,"data_stored":657370024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566240,"kb_used_data":245776,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93805600,"statfs":{"total":96636764160,"available":96056934400,"internally_reserved":0,"allocated":251674624,"data_stored":251392796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,2,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790041,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554988,"kb_used_data":167900,"kb_used_omap":4,"kb_used_meta":387067,"kb_avail":93816852,"statfs":{"total":96636764160,"available":96068456448,"internally_reserved":0,"allocated":171929600,"data_stored":171607565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":396357295},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638444,"kb_used_data":231964,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93733396,"statfs":{"total":96636764160,"available":95982997504,"internally_reserved":0,"allocated":237531136,"data_stored":237249303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,2,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235663360,"data_stored":235627230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171270144,"data_stored":171193812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":250589184,"data_stored":250548982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:45.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:45.667 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:45.669 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:45.688 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-11T01:22:44.609098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":226288393,"num_objects":55258,"num_object_clones":0,"num_object_copies":110516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55258,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":229987,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11336,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1759672,"kb_used_data":645640,"kb_used_omap":12,"kb_used_meta":1113971,"kb_avail":281355848,"statfs":{"total":289910292480,"available":288108388352,"internally_reserved":0,"allocated":661135360,"data_stored":660249664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1140706311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,9,4,4,5,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24420352,"num_objects":-5962,"num_object_clones":0,"num_object_copies":-11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-212,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008270"},"pg_stats":[{"pgid":"2.f","version":"43'16785","reported_seq":17009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.195015+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:43.195015+0000","last_peered":"2024-09-11T01:22:43.195015+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:43.195015+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'16977","reported_seq":17058,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:40.999714+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:40.999714+0000","last_peered":"2024-09-11T01:22:40.999714+0000","last_clean":"2024-09-11T01:22:40.999714+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:40.999714+0000","last_undegraded":"2024-09-11T01:22:40.999714+0000","last_fullsized":"2024-09-11T01:22:40.999714+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14521286,"num_objects":3547,"num_object_clones":0,"num_object_copies":7094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3547,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15075,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16644","reported_seq":16813,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.859193+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:40.859193+0000","last_peered":"2024-09-11T01:22:40.859193+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:40.859193+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13930520,"num_objects":3402,"num_object_clones":0,"num_object_copies":6804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3402,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14705,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-561}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16845","reported_seq":16891,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:41.007550+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:41.007550+0000","last_peered":"2024-09-11T01:22:41.007550+0000","last_clean":"2024-09-11T01:22:41.007550+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:41.007550+0000","last_undegraded":"2024-09-11T01:22:41.007550+0000","last_fullsized":"2024-09-11T01:22:41.007550+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14405632,"num_objects":3517,"num_object_clones":0,"num_object_copies":7034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16845,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16891","reported_seq":17011,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:43.219407+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:43.219407+0000","last_peered":"2024-09-11T01:22:43.219407+0000","last_clean":"2024-09-11T01:22:43.219407+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:43.219407+0000","last_undegraded":"2024-09-11T01:22:43.219407+0000","last_fullsized":"2024-09-11T01:22:43.219407+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869539,"num_objects":3387,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3387,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13689,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16886","reported_seq":17151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.133181+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:43.133181+0000","last_peered":"2024-09-11T01:22:43.133181+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:43.133181+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13603299,"num_objects":3322,"num_object_clones":0,"num_object_copies":6644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13374,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-727},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17319","reported_seq":17607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.188612+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:43.188612+0000","last_peered":"2024-09-11T01:22:43.188612+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:43.188612+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14308294,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14072,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-711},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16974","reported_seq":17094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:43.141230+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:43.141230+0000","last_peered":"2024-09-11T01:22:43.141230+0000","last_clean":"2024-09-11T01:22:43.141230+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:43.141230+0000","last_undegraded":"2024-09-11T01:22:43.141230+0000","last_fullsized":"2024-09-11T01:22:43.141230+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14131200,"num_objects":3450,"num_object_clones":0,"num_object_copies":6900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13927,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16812","reported_seq":25156,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:40.987903+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:40.987903+0000","last_peered":"2024-09-11T01:22:40.987903+0000","last_clean":"2024-09-11T01:22:40.987903+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:40.987903+0000","last_undegraded":"2024-09-11T01:22:40.987903+0000","last_fullsized":"2024-09-11T01:22:40.987903+0000","mapping_epoch":40,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14078435,"num_objects":3438,"num_object_clones":0,"num_object_copies":6876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14679,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.676997+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.676997+0000","last_peered":"2024-09-11T01:20:45.676997+0000","last_clean":"2024-09-11T01:20:45.676997+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.676997+0000","last_undegraded":"2024-09-11T01:20:45.676997+0000","last_fullsized":"2024-09-11T01:20:45.676997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17090","reported_seq":17370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.174282+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:43.174282+0000","last_peered":"2024-09-11T01:22:43.174282+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:43.174282+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":3504,"num_object_clones":0,"num_object_copies":7008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13454,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16639","reported_seq":21673,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:41.013739+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:41.013739+0000","last_peered":"2024-09-11T01:22:41.013739+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:41.013739+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6600","ondisk_log_start":"39'6600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14336483,"num_objects":3501,"num_object_clones":0,"num_object_copies":7002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1559,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3501,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14408,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1942}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17043","reported_seq":17198,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.937720+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:40.937720+0000","last_peered":"2024-09-11T01:22:40.937720+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:40.937720+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14413824,"num_objects":3519,"num_object_clones":0,"num_object_copies":7038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-628}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'16885","reported_seq":25355,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:41.090896+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:41.090896+0000","last_peered":"2024-09-11T01:22:41.090896+0000","last_clean":"2024-09-11T01:22:41.090896+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:41.090896+0000","last_undegraded":"2024-09-11T01:22:41.090896+0000","last_fullsized":"2024-09-11T01:22:41.090896+0000","mapping_epoch":42,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13575,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17219","reported_seq":17504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.175858+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:43.175858+0000","last_peered":"2024-09-11T01:22:43.175858+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:43.175858+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":61,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677337+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677337+0000","last_peered":"2024-09-11T01:20:45.677337+0000","last_clean":"2024-09-11T01:20:45.677337+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677337+0000","last_undegraded":"2024-09-11T01:20:45.677337+0000","last_fullsized":"2024-09-11T01:20:45.677337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17134","reported_seq":17431,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.253706+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:43.253706+0000","last_peered":"2024-09-11T01:22:43.253706+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:43.253706+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14196736,"num_objects":3466,"num_object_clones":0,"num_object_copies":6932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13669,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16779","reported_seq":17019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.236855+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:43.236855+0000","last_peered":"2024-09-11T01:22:43.236855+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:43.236855+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13759430,"num_objects":3361,"num_object_clones":0,"num_object_copies":6722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13436,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-741},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":226288374,"num_objects":55256,"num_object_clones":0,"num_object_copies":110512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55256,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":229985,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11335,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":657522688,"data_stored":657370024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160922,"ondisk_log_size":160922,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566240,"kb_used_data":245776,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93805600,"statfs":{"total":96636764160,"available":96056934400,"internally_reserved":0,"allocated":251674624,"data_stored":251392796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,2,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790041,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554988,"kb_used_data":167900,"kb_used_omap":4,"kb_used_meta":387067,"kb_avail":93816852,"statfs":{"total":96636764160,"available":96068456448,"internally_reserved":0,"allocated":171929600,"data_stored":171607565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":396357295},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638444,"kb_used_data":231964,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93733396,"statfs":{"total":96636764160,"available":95982997504,"internally_reserved":0,"allocated":237531136,"data_stored":237249303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,2,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235663360,"data_stored":235627230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":171270144,"data_stored":171193812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":250589184,"data_stored":250548982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'16785", 'reported_seq': 17009, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.195015+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:43.195015+0000', 'last_peered': '2024-09-11T01:22:43.195015+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:43.195015+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14143488, 'num_objects': 3453, 'num_object_clones': 0, 'num_object_copies': 6906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16785, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -595}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'16644", 'reported_seq': 16813, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:40.859193+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:40.859193+0000', 'last_peered': '2024-09-11T01:22:40.859193+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:40.859193+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13930520, 'num_objects': 3402, 'num_object_clones': 0, 'num_object_copies': 6804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3402, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14705, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -561}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'16886", 'reported_seq': 17151, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.133181+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:43.133181+0000', 'last_peered': '2024-09-11T01:22:43.133181+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:43.133181+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'6800", 'ondisk_log_start': "39'6800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13603299, 'num_objects': 3322, 'num_object_clones': 0, 'num_object_copies': 6644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3322, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13374, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -727}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'17319", 'reported_seq': 17607, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.188612+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:43.188612+0000', 'last_peered': '2024-09-11T01:22:43.188612+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:43.188612+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14308294, 'num_objects': 3495, 'num_object_clones': 0, 'num_object_copies': 6990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3495, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14072, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -711}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:45.690 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'17090", 'reported_seq': 17370, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.174282+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:43.174282+0000', 'last_peered': '2024-09-11T01:22:43.174282+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:43.174282+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14352384, 'num_objects': 3504, 'num_object_clones': 0, 'num_object_copies': 7008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13454, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -526}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'16639", 'reported_seq': 21673, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:41.013739+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:41.013739+0000', 'last_peered': '2024-09-11T01:22:41.013739+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:41.013739+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'6600", 'ondisk_log_start': "39'6600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14336483, 'num_objects': 3501, 'num_object_clones': 0, 'num_object_copies': 7002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1559, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3501, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14408, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2469, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1942}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'17043", 'reported_seq': 17198, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:40.937720+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:40.937720+0000', 'last_peered': '2024-09-11T01:22:40.937720+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:40.937720+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14413824, 'num_objects': 3519, 'num_object_clones': 0, 'num_object_copies': 7038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14680, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -628}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17219", 'reported_seq': 17504, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.175858+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:43.175858+0000', 'last_peered': '2024-09-11T01:22:43.175858+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:43.175858+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7200", 'ondisk_log_start': "39'7200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14036992, 'num_objects': 3427, 'num_object_clones': 0, 'num_object_copies': 6854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3427, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13612, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -690}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:45.691 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'17134", 'reported_seq': 17431, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.253706+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:43.253706+0000', 'last_peered': '2024-09-11T01:22:43.253706+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:43.253706+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'7100", 'ondisk_log_start': "39'7100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14196736, 'num_objects': 3466, 'num_object_clones': 0, 'num_object_copies': 6932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13669, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -599}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.692 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:45.692 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'16779", 'reported_seq': 17019, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:43.236855+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:43.236855+0000', 'last_peered': '2024-09-11T01:22:43.236855+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:43.236855+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13759430, 'num_objects': 3361, 'num_object_clones': 0, 'num_object_copies': 6722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3361, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13436, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -741}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:45.692 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:46.427 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:46.427 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38299/110516 objects degraded (34.655%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":566,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":55258,"data_bytes":226288393,"bytes_used":1801904128,"bytes_avail":288108388352,"bytes_total":289910292480,"degraded_objects":38299,"degraded_total":110516,"degraded_ratio":0.34654710630135005,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":212,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:46.447 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:46.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:47.030 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:47.033 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:47.058 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":320,"stamp":"2024-09-11T01:22:46.610027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":222126857,"num_objects":54242,"num_object_clones":0,"num_object_copies":108484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54242,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":231003,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160941,"ondisk_log_size":160941,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1739684,"kb_used_data":633972,"kb_used_omap":12,"kb_used_meta":1105651,"kb_avail":281375836,"statfs":{"total":289910292480,"available":288128856064,"internally_reserved":0,"allocated":649187328,"data_stored":648301632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1132186631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,3,4,8,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23138304,"num_objects":-5649,"num_object_clones":0,"num_object_copies":-11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5649,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008499"},"pg_stats":[{"pgid":"2.f","version":"43'16785","reported_seq":17009,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.195015+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:43.195015+0000","last_peered":"2024-09-11T01:22:43.195015+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:43.195015+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-595}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17129","reported_seq":17210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.995086+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:45.995086+0000","last_peered":"2024-09-11T01:22:45.995086+0000","last_clean":"2024-09-11T01:22:45.995086+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:45.995086+0000","last_undegraded":"2024-09-11T01:22:45.995086+0000","last_fullsized":"2024-09-11T01:22:45.995086+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13898694,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15227,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16776","reported_seq":16945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.988880+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:45.988880+0000","last_peered":"2024-09-11T01:22:45.988880+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:45.988880+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389848,"num_objects":3270,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14837,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16971","reported_seq":17017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.924464+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:45.924464+0000","last_peered":"2024-09-11T01:22:45.924464+0000","last_clean":"2024-09-11T01:22:45.924464+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:45.924464+0000","last_undegraded":"2024-09-11T01:22:45.924464+0000","last_fullsized":"2024-09-11T01:22:45.924464+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13889536,"num_objects":3391,"num_object_clones":0,"num_object_copies":6782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16971,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'16891","reported_seq":17011,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:43.219407+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:43.219407+0000","last_peered":"2024-09-11T01:22:43.219407+0000","last_clean":"2024-09-11T01:22:43.219407+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:43.219407+0000","last_undegraded":"2024-09-11T01:22:43.219407+0000","last_fullsized":"2024-09-11T01:22:43.219407+0000","mapping_epoch":31,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869539,"num_objects":3387,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3387,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13689,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16886","reported_seq":17151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.133181+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:43.133181+0000","last_peered":"2024-09-11T01:22:43.133181+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:43.133181+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6800","ondisk_log_start":"39'6800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13603299,"num_objects":3322,"num_object_clones":0,"num_object_copies":6644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13374,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-727},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17319","reported_seq":17607,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.188612+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:43.188612+0000","last_peered":"2024-09-11T01:22:43.188612+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:43.188612+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14308294,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14072,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-711},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'16974","reported_seq":17094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:43.141230+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:43.141230+0000","last_peered":"2024-09-11T01:22:43.141230+0000","last_clean":"2024-09-11T01:22:43.141230+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:43.141230+0000","last_undegraded":"2024-09-11T01:22:43.141230+0000","last_fullsized":"2024-09-11T01:22:43.141230+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14131200,"num_objects":3450,"num_object_clones":0,"num_object_copies":6900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13927,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16962","reported_seq":25306,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.258814+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:46.258814+0000","last_peered":"2024-09-11T01:22:46.258814+0000","last_clean":"2024-09-11T01:22:46.258814+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:46.258814+0000","last_undegraded":"2024-09-11T01:22:46.258814+0000","last_fullsized":"2024-09-11T01:22:46.258814+0000","mapping_epoch":40,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13464035,"num_objects":3288,"num_object_clones":0,"num_object_copies":6576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14829,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17090","reported_seq":17370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.174282+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:43.174282+0000","last_peered":"2024-09-11T01:22:43.174282+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:43.174282+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":3504,"num_object_clones":0,"num_object_copies":7008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13454,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16785","reported_seq":22000,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:46.012821+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:46.012821+0000","last_peered":"2024-09-11T01:22:46.012821+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:46.012821+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13738467,"num_objects":3355,"num_object_clones":0,"num_object_copies":6710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14554,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2560,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17210","reported_seq":17365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.973407+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:45.973407+0000","last_peered":"2024-09-11T01:22:45.973407+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:45.973407+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13729792,"num_objects":3352,"num_object_clones":0,"num_object_copies":6704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17028","reported_seq":25499,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.140870+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:46.140870+0000","last_peered":"2024-09-11T01:22:46.140870+0000","last_clean":"2024-09-11T01:22:46.140870+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:46.140870+0000","last_undegraded":"2024-09-11T01:22:46.140870+0000","last_fullsized":"2024-09-11T01:22:46.140870+0000","mapping_epoch":42,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13615104,"num_objects":3324,"num_object_clones":0,"num_object_copies":6648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17219","reported_seq":17504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.175858+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:43.175858+0000","last_peered":"2024-09-11T01:22:43.175858+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:43.175858+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17134","reported_seq":17431,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.253706+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:43.253706+0000","last_peered":"2024-09-11T01:22:43.253706+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:43.253706+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14196736,"num_objects":3466,"num_object_clones":0,"num_object_copies":6932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13669,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-599}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16779","reported_seq":17019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:43.236855+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:43.236855+0000","last_peered":"2024-09-11T01:22:43.236855+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:43.236855+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13759430,"num_objects":3361,"num_object_clones":0,"num_object_copies":6722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13436,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-741},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":222126838,"num_objects":54240,"num_object_clones":0,"num_object_copies":108480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54240,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":231001,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":645357568,"data_stored":645204904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565760,"kb_used_data":241136,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93806080,"statfs":{"total":96636764160,"available":96057425920,"internally_reserved":0,"allocated":246923264,"data_stored":246641436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790042,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535480,"kb_used_data":160872,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93836360,"statfs":{"total":96636764160,"available":96088432640,"internally_reserved":0,"allocated":164732928,"data_stored":164410893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638444,"kb_used_data":231964,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93733396,"statfs":{"total":96636764160,"available":95982997504,"internally_reserved":0,"allocated":237531136,"data_stored":237249303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,2,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":235663360,"data_stored":235627230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163872768,"data_stored":163796436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245821440,"data_stored":245781238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:50.060 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:50.622 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:50.624 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:50.644 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-11T01:22:50.611571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":216859401,"num_objects":52956,"num_object_clones":0,"num_object_copies":105912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52956,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":232289,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1737004,"kb_used_data":627132,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281378516,"statfs":{"total":289910292480,"available":288131600384,"internally_reserved":0,"allocated":642183168,"data_stored":641297472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1136446471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,10,11,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19070976,"num_objects":-4656,"num_object_clones":0,"num_object_copies":-9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005049"},"pg_stats":[{"pgid":"2.f","version":"43'16925","reported_seq":17148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.250625+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:48.250625+0000","last_peered":"2024-09-11T01:22:48.250625+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:48.250625+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16925,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17129","reported_seq":17210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.995086+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:45.995086+0000","last_peered":"2024-09-11T01:22:45.995086+0000","last_clean":"2024-09-11T01:22:45.995086+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:45.995086+0000","last_undegraded":"2024-09-11T01:22:45.995086+0000","last_fullsized":"2024-09-11T01:22:45.995086+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13898694,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15227,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16776","reported_seq":16945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.988880+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:45.988880+0000","last_peered":"2024-09-11T01:22:45.988880+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:45.988880+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389848,"num_objects":3270,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14837,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16971","reported_seq":17017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.924464+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:45.924464+0000","last_peered":"2024-09-11T01:22:45.924464+0000","last_clean":"2024-09-11T01:22:45.924464+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:45.924464+0000","last_undegraded":"2024-09-11T01:22:45.924464+0000","last_fullsized":"2024-09-11T01:22:45.924464+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13889536,"num_objects":3391,"num_object_clones":0,"num_object_copies":6782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16971,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17023","reported_seq":17145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:48.220791+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:48.220791+0000","last_peered":"2024-09-11T01:22:48.220791+0000","last_clean":"2024-09-11T01:22:48.220791+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:48.220791+0000","last_undegraded":"2024-09-11T01:22:48.220791+0000","last_fullsized":"2024-09-11T01:22:48.220791+0000","mapping_epoch":31,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13328867,"num_objects":3255,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13821,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16998","reported_seq":17265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.266742+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:48.266742+0000","last_peered":"2024-09-11T01:22:48.266742+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:48.266742+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13144547,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13486,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-839},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17478","reported_seq":17766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.172684+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:48.172684+0000","last_peered":"2024-09-11T01:22:48.172684+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:48.172684+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13657030,"num_objects":3336,"num_object_clones":0,"num_object_copies":6672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3336,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14231,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-870},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17135","reported_seq":17255,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:48.220121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:48.220121+0000","last_peered":"2024-09-11T01:22:48.220121+0000","last_clean":"2024-09-11T01:22:48.220121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:48.220121+0000","last_undegraded":"2024-09-11T01:22:48.220121+0000","last_fullsized":"2024-09-11T01:22:48.220121+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14088,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16962","reported_seq":25306,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.258814+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:46.258814+0000","last_peered":"2024-09-11T01:22:46.258814+0000","last_clean":"2024-09-11T01:22:46.258814+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:46.258814+0000","last_undegraded":"2024-09-11T01:22:46.258814+0000","last_fullsized":"2024-09-11T01:22:46.258814+0000","mapping_epoch":40,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13464035,"num_objects":3288,"num_object_clones":0,"num_object_copies":6576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14829,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17243","reported_seq":17523,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.235560+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:48.235560+0000","last_peered":"2024-09-11T01:22:48.235560+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:48.235560+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13725696,"num_objects":3351,"num_object_clones":0,"num_object_copies":6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13607,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16785","reported_seq":22000,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:46.012821+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:46.012821+0000","last_peered":"2024-09-11T01:22:46.012821+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:46.012821+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13738467,"num_objects":3355,"num_object_clones":0,"num_object_copies":6710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14554,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2560,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17210","reported_seq":17365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.973407+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:45.973407+0000","last_peered":"2024-09-11T01:22:45.973407+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:45.973407+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13729792,"num_objects":3352,"num_object_clones":0,"num_object_copies":6704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17028","reported_seq":25499,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.140870+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:46.140870+0000","last_peered":"2024-09-11T01:22:46.140870+0000","last_clean":"2024-09-11T01:22:46.140870+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:46.140870+0000","last_undegraded":"2024-09-11T01:22:46.140870+0000","last_fullsized":"2024-09-11T01:22:46.140870+0000","mapping_epoch":42,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13615104,"num_objects":3324,"num_object_clones":0,"num_object_copies":6648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17357","reported_seq":17642,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.279392+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:48.279392+0000","last_peered":"2024-09-11T01:22:48.279392+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:48.279392+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-828}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17292","reported_seq":17593,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.247586+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:48.247586+0000","last_peered":"2024-09-11T01:22:48.247586+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:48.247586+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16912","reported_seq":17152,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.256576+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:48.256576+0000","last_peered":"2024-09-11T01:22:48.256576+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:48.256576+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13214662,"num_objects":3228,"num_object_clones":0,"num_object_copies":6456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13569,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":216859382,"num_objects":52954,"num_object_clones":0,"num_object_copies":105908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52954,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":232287,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638410752,"data_stored":638258088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565760,"kb_used_data":241136,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93806080,"statfs":{"total":96636764160,"available":96057425920,"internally_reserved":0,"allocated":246923264,"data_stored":246641436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790042,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535480,"kb_used_data":160872,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93836360,"statfs":{"total":96636764160,"available":96088432640,"internally_reserved":0,"allocated":164732928,"data_stored":164410893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635764,"kb_used_data":225124,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93736076,"statfs":{"total":96636764160,"available":95985741824,"internally_reserved":0,"allocated":230526976,"data_stored":230245143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228716544,"data_stored":228680414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163872768,"data_stored":163796436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245821440,"data_stored":245781238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:50.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:51.260 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:51.262 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:51.294 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-11T01:22:50.611571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":216859401,"num_objects":52956,"num_object_clones":0,"num_object_copies":105912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52956,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":232289,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1737004,"kb_used_data":627132,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281378516,"statfs":{"total":289910292480,"available":288131600384,"internally_reserved":0,"allocated":642183168,"data_stored":641297472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1136446471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,10,11,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19070976,"num_objects":-4656,"num_object_clones":0,"num_object_copies":-9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005049"},"pg_stats":[{"pgid":"2.f","version":"43'16925","reported_seq":17148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.250625+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:48.250625+0000","last_peered":"2024-09-11T01:22:48.250625+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:48.250625+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16925,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17129","reported_seq":17210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.995086+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:45.995086+0000","last_peered":"2024-09-11T01:22:45.995086+0000","last_clean":"2024-09-11T01:22:45.995086+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:45.995086+0000","last_undegraded":"2024-09-11T01:22:45.995086+0000","last_fullsized":"2024-09-11T01:22:45.995086+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13898694,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15227,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16776","reported_seq":16945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.988880+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:45.988880+0000","last_peered":"2024-09-11T01:22:45.988880+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:45.988880+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389848,"num_objects":3270,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14837,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16971","reported_seq":17017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.924464+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:45.924464+0000","last_peered":"2024-09-11T01:22:45.924464+0000","last_clean":"2024-09-11T01:22:45.924464+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:45.924464+0000","last_undegraded":"2024-09-11T01:22:45.924464+0000","last_fullsized":"2024-09-11T01:22:45.924464+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13889536,"num_objects":3391,"num_object_clones":0,"num_object_copies":6782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16971,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17023","reported_seq":17145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:48.220791+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:48.220791+0000","last_peered":"2024-09-11T01:22:48.220791+0000","last_clean":"2024-09-11T01:22:48.220791+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:48.220791+0000","last_undegraded":"2024-09-11T01:22:48.220791+0000","last_fullsized":"2024-09-11T01:22:48.220791+0000","mapping_epoch":31,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13328867,"num_objects":3255,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13821,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16998","reported_seq":17265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.266742+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:48.266742+0000","last_peered":"2024-09-11T01:22:48.266742+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:48.266742+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13144547,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13486,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-839},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17478","reported_seq":17766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.172684+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:48.172684+0000","last_peered":"2024-09-11T01:22:48.172684+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:48.172684+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13657030,"num_objects":3336,"num_object_clones":0,"num_object_copies":6672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3336,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14231,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-870},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17135","reported_seq":17255,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:48.220121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:48.220121+0000","last_peered":"2024-09-11T01:22:48.220121+0000","last_clean":"2024-09-11T01:22:48.220121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:48.220121+0000","last_undegraded":"2024-09-11T01:22:48.220121+0000","last_fullsized":"2024-09-11T01:22:48.220121+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14088,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16962","reported_seq":25306,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.258814+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:46.258814+0000","last_peered":"2024-09-11T01:22:46.258814+0000","last_clean":"2024-09-11T01:22:46.258814+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:46.258814+0000","last_undegraded":"2024-09-11T01:22:46.258814+0000","last_fullsized":"2024-09-11T01:22:46.258814+0000","mapping_epoch":40,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13464035,"num_objects":3288,"num_object_clones":0,"num_object_copies":6576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14829,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17243","reported_seq":17523,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.235560+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:48.235560+0000","last_peered":"2024-09-11T01:22:48.235560+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:48.235560+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13725696,"num_objects":3351,"num_object_clones":0,"num_object_copies":6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13607,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16785","reported_seq":22000,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:46.012821+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:46.012821+0000","last_peered":"2024-09-11T01:22:46.012821+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:46.012821+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13738467,"num_objects":3355,"num_object_clones":0,"num_object_copies":6710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14554,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2560,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17210","reported_seq":17365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.973407+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:45.973407+0000","last_peered":"2024-09-11T01:22:45.973407+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:45.973407+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13729792,"num_objects":3352,"num_object_clones":0,"num_object_copies":6704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17028","reported_seq":25499,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.140870+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:46.140870+0000","last_peered":"2024-09-11T01:22:46.140870+0000","last_clean":"2024-09-11T01:22:46.140870+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:46.140870+0000","last_undegraded":"2024-09-11T01:22:46.140870+0000","last_fullsized":"2024-09-11T01:22:46.140870+0000","mapping_epoch":42,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13615104,"num_objects":3324,"num_object_clones":0,"num_object_copies":6648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17357","reported_seq":17642,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.279392+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:48.279392+0000","last_peered":"2024-09-11T01:22:48.279392+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:48.279392+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-828}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17292","reported_seq":17593,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.247586+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:48.247586+0000","last_peered":"2024-09-11T01:22:48.247586+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:48.247586+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16912","reported_seq":17152,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.256576+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:48.256576+0000","last_peered":"2024-09-11T01:22:48.256576+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:48.256576+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13214662,"num_objects":3228,"num_object_clones":0,"num_object_copies":6456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13569,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":216859382,"num_objects":52954,"num_object_clones":0,"num_object_copies":105908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52954,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":232287,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638410752,"data_stored":638258088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565760,"kb_used_data":241136,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93806080,"statfs":{"total":96636764160,"available":96057425920,"internally_reserved":0,"allocated":246923264,"data_stored":246641436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790042,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535480,"kb_used_data":160872,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93836360,"statfs":{"total":96636764160,"available":96088432640,"internally_reserved":0,"allocated":164732928,"data_stored":164410893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635764,"kb_used_data":225124,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93736076,"statfs":{"total":96636764160,"available":95985741824,"internally_reserved":0,"allocated":230526976,"data_stored":230245143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228716544,"data_stored":228680414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163872768,"data_stored":163796436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245821440,"data_stored":245781238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:51.295 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'16925", 'reported_seq': 17148, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.250625+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:48.250625+0000', 'last_peered': '2024-09-11T01:22:48.250625+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:48.250625+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'6900", 'ondisk_log_start': "39'6900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13570048, 'num_objects': 3313, 'num_object_clones': 0, 'num_object_copies': 6626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16925, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -735}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'16776", 'reported_seq': 16945, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:45.988880+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:45.988880+0000', 'last_peered': '2024-09-11T01:22:45.988880+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:45.988880+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13389848, 'num_objects': 3270, 'num_object_clones': 0, 'num_object_copies': 6540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3270, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14837, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'16998", 'reported_seq': 17265, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.266742+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:48.266742+0000', 'last_peered': '2024-09-11T01:22:48.266742+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:48.266742+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'6900", 'ondisk_log_start': "39'6900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13144547, 'num_objects': 3210, 'num_object_clones': 0, 'num_object_copies': 6420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3210, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13486, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -839}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'17478", 'reported_seq': 17766, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.172684+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:48.172684+0000', 'last_peered': '2024-09-11T01:22:48.172684+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:48.172684+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'7400", 'ondisk_log_start': "39'7400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13657030, 'num_objects': 3336, 'num_object_clones': 0, 'num_object_copies': 6672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3336, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14231, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -870}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:51.296 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'17243", 'reported_seq': 17523, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.235560+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:48.235560+0000', 'last_peered': '2024-09-11T01:22:48.235560+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:48.235560+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'7200", 'ondisk_log_start': "39'7200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13725696, 'num_objects': 3351, 'num_object_clones': 0, 'num_object_copies': 6702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3351, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13607, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -679}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'16785", 'reported_seq': 22000, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:46.012821+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:46.012821+0000', 'last_peered': '2024-09-11T01:22:46.012821+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:46.012821+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'6700", 'ondisk_log_start': "39'6700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13738467, 'num_objects': 3355, 'num_object_clones': 0, 'num_object_copies': 6710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1468, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3355, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14554, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2560, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1887}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'17210", 'reported_seq': 17365, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:45.973407+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:45.973407+0000', 'last_peered': '2024-09-11T01:22:45.973407+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:45.973407+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'7200", 'ondisk_log_start': "39'7200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13729792, 'num_objects': 3352, 'num_object_clones': 0, 'num_object_copies': 6704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14847, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -795}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17357", 'reported_seq': 17642, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.279392+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:48.279392+0000', 'last_peered': '2024-09-11T01:22:48.279392+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:48.279392+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13471744, 'num_objects': 3289, 'num_object_clones': 0, 'num_object_copies': 6578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13750, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -828}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'17292", 'reported_seq': 17593, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.247586+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:48.247586+0000', 'last_peered': '2024-09-11T01:22:48.247586+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:48.247586+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'7200", 'ondisk_log_start': "39'7200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13549568, 'num_objects': 3308, 'num_object_clones': 0, 'num_object_copies': 6616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13827, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -757}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:51.298 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'16912", 'reported_seq': 17152, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:48.256576+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:48.256576+0000', 'last_peered': '2024-09-11T01:22:48.256576+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:48.256576+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'6900", 'ondisk_log_start': "39'6900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13214662, 'num_objects': 3228, 'num_object_clones': 0, 'num_object_copies': 6456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3228, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13569, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -874}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:51.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:51.992 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:51.992 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38208/105912 objects degraded (36.075%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":572,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":52956,"data_bytes":216859401,"bytes_used":1778692096,"bytes_avail":288131600384,"bytes_total":289910292480,"degraded_objects":38208,"degraded_total":105912,"degraded_ratio":0.36075232268298207,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:52.019 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:52.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:52.590 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:52.590 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:52.620 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-11T01:22:50.611571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":216859401,"num_objects":52956,"num_object_clones":0,"num_object_copies":105912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52956,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":232289,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11427,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1737004,"kb_used_data":627132,"kb_used_omap":12,"kb_used_meta":1109811,"kb_avail":281378516,"statfs":{"total":289910292480,"available":288131600384,"internally_reserved":0,"allocated":642183168,"data_stored":641297472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1136446471},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,10,11,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19070976,"num_objects":-4656,"num_object_clones":0,"num_object_copies":-9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4656,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005049"},"pg_stats":[{"pgid":"2.f","version":"43'16925","reported_seq":17148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.250625+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:48.250625+0000","last_peered":"2024-09-11T01:22:48.250625+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:48.250625+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13570048,"num_objects":3313,"num_object_clones":0,"num_object_copies":6626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16925,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17129","reported_seq":17210,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.995086+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:45.995086+0000","last_peered":"2024-09-11T01:22:45.995086+0000","last_clean":"2024-09-11T01:22:45.995086+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:45.995086+0000","last_undegraded":"2024-09-11T01:22:45.995086+0000","last_fullsized":"2024-09-11T01:22:45.995086+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13898694,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15227,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16776","reported_seq":16945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.988880+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:45.988880+0000","last_peered":"2024-09-11T01:22:45.988880+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:45.988880+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13389848,"num_objects":3270,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14837,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-693}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'16971","reported_seq":17017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:45.924464+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:45.924464+0000","last_peered":"2024-09-11T01:22:45.924464+0000","last_clean":"2024-09-11T01:22:45.924464+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:45.924464+0000","last_undegraded":"2024-09-11T01:22:45.924464+0000","last_fullsized":"2024-09-11T01:22:45.924464+0000","mapping_epoch":31,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13889536,"num_objects":3391,"num_object_clones":0,"num_object_copies":6782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16971,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17023","reported_seq":17145,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:48.220791+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:48.220791+0000","last_peered":"2024-09-11T01:22:48.220791+0000","last_clean":"2024-09-11T01:22:48.220791+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:48.220791+0000","last_undegraded":"2024-09-11T01:22:48.220791+0000","last_fullsized":"2024-09-11T01:22:48.220791+0000","mapping_epoch":31,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13328867,"num_objects":3255,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13821,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'16998","reported_seq":17265,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.266742+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:48.266742+0000","last_peered":"2024-09-11T01:22:48.266742+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:48.266742+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13144547,"num_objects":3210,"num_object_clones":0,"num_object_copies":6420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13486,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-839},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17478","reported_seq":17766,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.172684+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:48.172684+0000","last_peered":"2024-09-11T01:22:48.172684+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:48.172684+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13657030,"num_objects":3336,"num_object_clones":0,"num_object_copies":6672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3336,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14231,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-870},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17135","reported_seq":17255,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:48.220121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:48.220121+0000","last_peered":"2024-09-11T01:22:48.220121+0000","last_clean":"2024-09-11T01:22:48.220121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:48.220121+0000","last_undegraded":"2024-09-11T01:22:48.220121+0000","last_fullsized":"2024-09-11T01:22:48.220121+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14088,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'16962","reported_seq":25306,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.258814+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:46.258814+0000","last_peered":"2024-09-11T01:22:46.258814+0000","last_clean":"2024-09-11T01:22:46.258814+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:46.258814+0000","last_undegraded":"2024-09-11T01:22:46.258814+0000","last_fullsized":"2024-09-11T01:22:46.258814+0000","mapping_epoch":40,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13464035,"num_objects":3288,"num_object_clones":0,"num_object_copies":6576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14829,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17243","reported_seq":17523,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.235560+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:48.235560+0000","last_peered":"2024-09-11T01:22:48.235560+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:48.235560+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13725696,"num_objects":3351,"num_object_clones":0,"num_object_copies":6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13607,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-679}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16785","reported_seq":22000,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:46.012821+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:46.012821+0000","last_peered":"2024-09-11T01:22:46.012821+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:46.012821+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6700","ondisk_log_start":"39'6700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13738467,"num_objects":3355,"num_object_clones":0,"num_object_copies":6710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3355,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14554,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2560,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17210","reported_seq":17365,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.973407+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:45.973407+0000","last_peered":"2024-09-11T01:22:45.973407+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:45.973407+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13729792,"num_objects":3352,"num_object_clones":0,"num_object_copies":6704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14847,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-795}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17028","reported_seq":25499,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:46.140870+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:46.140870+0000","last_peered":"2024-09-11T01:22:46.140870+0000","last_clean":"2024-09-11T01:22:46.140870+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:46.140870+0000","last_undegraded":"2024-09-11T01:22:46.140870+0000","last_fullsized":"2024-09-11T01:22:46.140870+0000","mapping_epoch":42,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13615104,"num_objects":3324,"num_object_clones":0,"num_object_copies":6648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17357","reported_seq":17642,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.279392+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:48.279392+0000","last_peered":"2024-09-11T01:22:48.279392+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:48.279392+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13750,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-828}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17292","reported_seq":17593,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.247586+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:48.247586+0000","last_peered":"2024-09-11T01:22:48.247586+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:48.247586+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13549568,"num_objects":3308,"num_object_clones":0,"num_object_copies":6616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13827,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-757}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'16912","reported_seq":17152,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:48.256576+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:48.256576+0000","last_peered":"2024-09-11T01:22:48.256576+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:48.256576+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13214662,"num_objects":3228,"num_object_clones":0,"num_object_copies":6456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3228,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13569,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-874},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":216859382,"num_objects":52954,"num_object_clones":0,"num_object_copies":105908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38208,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52954,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":232287,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11426,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":638410752,"data_stored":638258088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565760,"kb_used_data":241136,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93806080,"statfs":{"total":96636764160,"available":96057425920,"internally_reserved":0,"allocated":246923264,"data_stored":246641436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790042,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535480,"kb_used_data":160872,"kb_used_omap":4,"kb_used_meta":374587,"kb_avail":93836360,"statfs":{"total":96636764160,"available":96088432640,"internally_reserved":0,"allocated":164732928,"data_stored":164410893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383577775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635764,"kb_used_data":225124,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93736076,"statfs":{"total":96636764160,"available":95985741824,"internally_reserved":0,"allocated":230526976,"data_stored":230245143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":228716544,"data_stored":228680414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":163872768,"data_stored":163796436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245821440,"data_stored":245781238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:55.622 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:56.216 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:56.217 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:56.244 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-11T01:22:54.613351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":207368969,"num_objects":50639,"num_object_clones":0,"num_object_copies":101278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50639,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":234606,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11533,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160944,"ondisk_log_size":160944,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1771064,"kb_used_data":608448,"kb_used_omap":12,"kb_used_meta":1162547,"kb_avail":281344456,"statfs":{"total":289910292480,"available":288096722944,"internally_reserved":0,"allocated":623050752,"data_stored":622165056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1190448135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,6,11,2,9,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24469504,"num_objects":-5974,"num_object_clones":0,"num_object_copies":-11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-197,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5974,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005158"},"pg_stats":[{"pgid":"2.f","version":"43'17070","reported_seq":17294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.090695+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:53.090695+0000","last_peered":"2024-09-11T01:22:53.090695+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:53.090695+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":3168,"num_object_clones":0,"num_object_copies":6336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17070,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-880}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17260","reported_seq":17341,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:50.988774+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:50.988774+0000","last_peered":"2024-09-11T01:22:50.988774+0000","last_clean":"2024-09-11T01:22:50.988774+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:50.988774+0000","last_undegraded":"2024-09-11T01:22:50.988774+0000","last_fullsized":"2024-09-11T01:22:50.988774+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13362118,"num_objects":3264,"num_object_clones":0,"num_object_copies":6528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3264,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15358,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'16933","reported_seq":17102,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.923920+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:50.923920+0000","last_peered":"2024-09-11T01:22:50.923920+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:50.923920+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12746776,"num_objects":3113,"num_object_clones":0,"num_object_copies":6226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14994,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-850}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17129","reported_seq":17175,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:51.001606+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:51.001606+0000","last_peered":"2024-09-11T01:22:51.001606+0000","last_clean":"2024-09-11T01:22:51.001606+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:51.001606+0000","last_undegraded":"2024-09-11T01:22:51.001606+0000","last_fullsized":"2024-09-11T01:22:51.001606+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17129,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17158","reported_seq":17280,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:53.240627+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:53.240627+0000","last_peered":"2024-09-11T01:22:53.240627+0000","last_clean":"2024-09-11T01:22:53.240627+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:53.240627+0000","last_undegraded":"2024-09-11T01:22:53.240627+0000","last_fullsized":"2024-09-11T01:22:53.240627+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12775907,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13956,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17144","reported_seq":17411,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.213618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:53.213618+0000","last_peered":"2024-09-11T01:22:53.213618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:53.213618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12546531,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13632,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17621","reported_seq":17909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.313065+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:53.313065+0000","last_peered":"2024-09-11T01:22:53.313065+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:53.313065+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13071302,"num_objects":3193,"num_object_clones":0,"num_object_copies":6386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14374,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1013},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17293","reported_seq":17412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:53.298011+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:53.298011+0000","last_peered":"2024-09-11T01:22:53.298011+0000","last_clean":"2024-09-11T01:22:53.298011+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:53.298011+0000","last_undegraded":"2024-09-11T01:22:53.298011+0000","last_fullsized":"2024-09-11T01:22:53.298011+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12824576,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14246,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17100","reported_seq":25444,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:51.345988+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:51.345988+0000","last_peered":"2024-09-11T01:22:51.345988+0000","last_clean":"2024-09-11T01:22:51.345988+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:51.345988+0000","last_undegraded":"2024-09-11T01:22:51.345988+0000","last_fullsized":"2024-09-11T01:22:51.345988+0000","mapping_epoch":40,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898787,"num_objects":3150,"num_object_clones":0,"num_object_copies":6300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3150,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14967,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17393","reported_seq":17673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.250729+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:53.250729+0000","last_peered":"2024-09-11T01:22:53.250729+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:53.250729+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13111296,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13757,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'16945","reported_seq":22374,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:51.026836+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:51.026836+0000","last_peered":"2024-09-11T01:22:51.026836+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:51.026836+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'6900","ondisk_log_start":"39'6900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13083107,"num_objects":3195,"num_object_clones":0,"num_object_copies":6390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1362,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14714,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1833}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17368","reported_seq":17522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:51.025039+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:51.025039+0000","last_peered":"2024-09-11T01:22:51.025039+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:51.025039+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13082624,"num_objects":3194,"num_object_clones":0,"num_object_copies":6388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15005,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-953}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17177","reported_seq":25648,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:51.334947+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:51.334947+0000","last_peered":"2024-09-11T01:22:51.334947+0000","last_clean":"2024-09-11T01:22:51.334947+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:51.334947+0000","last_undegraded":"2024-09-11T01:22:51.334947+0000","last_fullsized":"2024-09-11T01:22:51.334947+0000","mapping_epoch":42,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13004800,"num_objects":3175,"num_object_clones":0,"num_object_copies":6350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13867,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17490","reported_seq":17775,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.300536+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:53.300536+0000","last_peered":"2024-09-11T01:22:53.300536+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:53.300536+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13883,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17419","reported_seq":17720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.256602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:53.256602+0000","last_peered":"2024-09-11T01:22:53.256602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:53.256602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13029376,"num_objects":3181,"num_object_clones":0,"num_object_copies":6362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17041","reported_seq":17281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.231923+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:53.231923+0000","last_peered":"2024-09-11T01:22:53.231923+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:53.231923+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12686278,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13698,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":207368950,"num_objects":50637,"num_object_clones":0,"num_object_copies":101274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50637,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":234604,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11532,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":619048960,"data_stored":618896296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160941,"ondisk_log_size":160941,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565276,"kb_used_data":236488,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93806564,"statfs":{"total":96636764160,"available":96057921536,"internally_reserved":0,"allocated":242163712,"data_stored":241881884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790043,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572884,"kb_used_data":153860,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93798956,"statfs":{"total":96636764160,"available":96050130944,"internally_reserved":0,"allocated":157552640,"data_stored":157230605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":429059759},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632904,"kb_used_data":218100,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93738936,"statfs":{"total":96636764160,"available":95988670464,"internally_reserved":0,"allocated":223334400,"data_stored":223052567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,5,0,1,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221585408,"data_stored":221549278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156561408,"data_stored":156485076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240902144,"data_stored":240861942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:56.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:56.843 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:56.848 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:56.867 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-11T01:22:56.614225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203051785,"num_objects":49585,"num_object_clones":0,"num_object_copies":99170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49585,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":235660,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11636,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769508,"kb_used_data":595760,"kb_used_omap":12,"kb_used_meta":1173683,"kb_avail":281346012,"statfs":{"total":289910292480,"available":288098316288,"internally_reserved":0,"allocated":610058240,"data_stored":609172544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1201851399},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,9,5,6,8,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23236608,"num_objects":-5673,"num_object_clones":0,"num_object_copies":-11346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5673,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005126"},"pg_stats":[{"pgid":"2.f","version":"43'17070","reported_seq":17294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.090695+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:53.090695+0000","last_peered":"2024-09-11T01:22:53.090695+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:53.090695+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":3168,"num_object_clones":0,"num_object_copies":6336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17070,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-880}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17429","reported_seq":17510,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.049556+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:56.049556+0000","last_peered":"2024-09-11T01:22:56.049556+0000","last_clean":"2024-09-11T01:22:56.049556+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:56.049556+0000","last_undegraded":"2024-09-11T01:22:56.049556+0000","last_fullsized":"2024-09-11T01:22:56.049556+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15527,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17076","reported_seq":17245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.967518+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:55.967518+0000","last_peered":"2024-09-11T01:22:55.967518+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:55.967518+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161048,"num_objects":2970,"num_object_clones":0,"num_object_copies":5940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17269","reported_seq":17315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.058289+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:56.058289+0000","last_peered":"2024-09-11T01:22:56.058289+0000","last_clean":"2024-09-11T01:22:56.058289+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:56.058289+0000","last_undegraded":"2024-09-11T01:22:56.058289+0000","last_fullsized":"2024-09-11T01:22:56.058289+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12668928,"num_objects":3093,"num_object_clones":0,"num_object_copies":6186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17269,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17158","reported_seq":17280,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:53.240627+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:53.240627+0000","last_peered":"2024-09-11T01:22:53.240627+0000","last_clean":"2024-09-11T01:22:53.240627+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:53.240627+0000","last_undegraded":"2024-09-11T01:22:53.240627+0000","last_fullsized":"2024-09-11T01:22:53.240627+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12775907,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13956,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17144","reported_seq":17411,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.213618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:53.213618+0000","last_peered":"2024-09-11T01:22:53.213618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:53.213618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12546531,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13632,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17621","reported_seq":17909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.313065+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:53.313065+0000","last_peered":"2024-09-11T01:22:53.313065+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:53.313065+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13071302,"num_objects":3193,"num_object_clones":0,"num_object_copies":6386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14374,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1013},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17293","reported_seq":17412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:53.298011+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:53.298011+0000","last_peered":"2024-09-11T01:22:53.298011+0000","last_clean":"2024-09-11T01:22:53.298011+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:53.298011+0000","last_undegraded":"2024-09-11T01:22:53.298011+0000","last_fullsized":"2024-09-11T01:22:53.298011+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12824576,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14246,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17265","reported_seq":25609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.454525+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:56.454525+0000","last_peered":"2024-09-11T01:22:56.454525+0000","last_clean":"2024-09-11T01:22:56.454525+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:56.454525+0000","last_undegraded":"2024-09-11T01:22:56.454525+0000","last_fullsized":"2024-09-11T01:22:56.454525+0000","mapping_epoch":40,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222947,"num_objects":2985,"num_object_clones":0,"num_object_copies":5970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15132,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17393","reported_seq":17673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.250729+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:53.250729+0000","last_peered":"2024-09-11T01:22:53.250729+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:53.250729+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13111296,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13757,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17079","reported_seq":22714,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.027640+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:56.027640+0000","last_peered":"2024-09-11T01:22:56.027640+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:56.027640+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534243,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14848,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2769,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17530","reported_seq":17685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.048198+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:56.048198+0000","last_peered":"2024-09-11T01:22:56.048198+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:56.048198+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15167,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17318","reported_seq":25790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.414555+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:56.414555+0000","last_peered":"2024-09-11T01:22:56.414555+0000","last_clean":"2024-09-11T01:22:56.414555+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:56.414555+0000","last_undegraded":"2024-09-11T01:22:56.414555+0000","last_fullsized":"2024-09-11T01:22:56.414555+0000","mapping_epoch":42,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12427264,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17490","reported_seq":17775,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.300536+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:53.300536+0000","last_peered":"2024-09-11T01:22:53.300536+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:53.300536+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13883,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17419","reported_seq":17720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.256602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:53.256602+0000","last_peered":"2024-09-11T01:22:53.256602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:53.256602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13029376,"num_objects":3181,"num_object_clones":0,"num_object_copies":6362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17041","reported_seq":17281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.231923+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:53.231923+0000","last_peered":"2024-09-11T01:22:53.231923+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:53.231923+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12686278,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13698,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203051766,"num_objects":49583,"num_object_clones":0,"num_object_copies":99166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49583,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":235658,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11635,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606945280,"data_stored":606792616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547196,"kb_used_data":230892,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93824644,"statfs":{"total":96636764160,"available":96076435456,"internally_reserved":0,"allocated":236433408,"data_stored":236151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589408,"kb_used_data":146768,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93782432,"statfs":{"total":96636764160,"available":96033210368,"internally_reserved":0,"allocated":150290432,"data_stored":149968397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632904,"kb_used_data":218100,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93738936,"statfs":{"total":96636764160,"available":95988670464,"internally_reserved":0,"allocated":223334400,"data_stored":223052567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,5,0,1,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221585408,"data_stored":221549278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149319680,"data_stored":149243348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236040192,"data_stored":235999990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17070", 'reported_seq': 17294, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.090695+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:53.090695+0000', 'last_peered': '2024-09-11T01:22:53.090695+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:53.090695+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12976128, 'num_objects': 3168, 'num_object_clones': 0, 'num_object_copies': 6336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17070, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -880}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17076", 'reported_seq': 17245, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.967518+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:55.967518+0000', 'last_peered': '2024-09-11T01:22:55.967518+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:55.967518+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12161048, 'num_objects': 2970, 'num_object_clones': 0, 'num_object_copies': 5940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2970, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15137, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -993}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17144", 'reported_seq': 17411, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.213618+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:53.213618+0000', 'last_peered': '2024-09-11T01:22:53.213618+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:53.213618+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7100", 'ondisk_log_start': "39'7100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12546531, 'num_objects': 3064, 'num_object_clones': 0, 'num_object_copies': 6128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3064, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13632, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -985}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.868 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'17621", 'reported_seq': 17909, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.313065+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:53.313065+0000', 'last_peered': '2024-09-11T01:22:53.313065+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:53.313065+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'7600", 'ondisk_log_start': "39'7600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13071302, 'num_objects': 3193, 'num_object_clones': 0, 'num_object_copies': 6386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3193, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14374, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1013}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'17393", 'reported_seq': 17673, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.250729+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:53.250729+0000', 'last_peered': '2024-09-11T01:22:53.250729+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:53.250729+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13111296, 'num_objects': 3201, 'num_object_clones': 0, 'num_object_copies': 6402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13757, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -829}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17079", 'reported_seq': 22714, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.027640+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:56.027640+0000', 'last_peered': '2024-09-11T01:22:56.027640+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:56.027640+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12534243, 'num_objects': 3061, 'num_object_clones': 0, 'num_object_copies': 6122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3061, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14848, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2769, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1802}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'17530", 'reported_seq': 17685, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.048198+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:56.048198+0000', 'last_peered': '2024-09-11T01:22:56.048198+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:56.048198+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12419072, 'num_objects': 3032, 'num_object_clones': 0, 'num_object_copies': 6064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15167, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1115}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17490", 'reported_seq': 17775, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.300536+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:53.300536+0000', 'last_peered': '2024-09-11T01:22:53.300536+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:53.300536+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7400", 'ondisk_log_start': "39'7400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12926976, 'num_objects': 3156, 'num_object_clones': 0, 'num_object_copies': 6312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13883, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -961}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'17419", 'reported_seq': 17720, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.256602+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:53.256602+0000', 'last_peered': '2024-09-11T01:22:53.256602+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:53.256602+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'7400", 'ondisk_log_start': "39'7400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13029376, 'num_objects': 3181, 'num_object_clones': 0, 'num_object_copies': 6362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13954, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:56.870 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17041", 'reported_seq': 17281, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:53.231923+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:53.231923+0000', 'last_peered': '2024-09-11T01:22:53.231923+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:53.231923+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12686278, 'num_objects': 3099, 'num_object_clones': 0, 'num_object_copies': 6198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3099, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13698, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1003}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.870 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:57.585 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:57.586 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37999/99170 objects degraded (38.317%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":578,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":49585,"data_bytes":203051785,"bytes_used":1811976192,"bytes_avail":288098316288,"bytes_total":289910292480,"degraded_objects":37999,"degraded_total":99170,"degraded_ratio":0.3831703136029041,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":472},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:57.613 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:57.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:58.188 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:22:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:22:58.211 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-09-11T01:22:56.614225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":203051785,"num_objects":49585,"num_object_clones":0,"num_object_copies":99170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49585,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":235660,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11636,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1769508,"kb_used_data":595760,"kb_used_omap":12,"kb_used_meta":1173683,"kb_avail":281346012,"statfs":{"total":289910292480,"available":288098316288,"internally_reserved":0,"allocated":610058240,"data_stored":609172544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1201851399},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,9,5,6,8,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23236608,"num_objects":-5673,"num_object_clones":0,"num_object_copies":-11346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5673,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005126"},"pg_stats":[{"pgid":"2.f","version":"43'17070","reported_seq":17294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.090695+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:53.090695+0000","last_peered":"2024-09-11T01:22:53.090695+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:53.090695+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12976128,"num_objects":3168,"num_object_clones":0,"num_object_copies":6336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17070,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-880}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17429","reported_seq":17510,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.049556+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:56.049556+0000","last_peered":"2024-09-11T01:22:56.049556+0000","last_clean":"2024-09-11T01:22:56.049556+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:56.049556+0000","last_undegraded":"2024-09-11T01:22:56.049556+0000","last_fullsized":"2024-09-11T01:22:56.049556+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15527,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17076","reported_seq":17245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.967518+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:55.967518+0000","last_peered":"2024-09-11T01:22:55.967518+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:55.967518+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161048,"num_objects":2970,"num_object_clones":0,"num_object_copies":5940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17269","reported_seq":17315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.058289+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:56.058289+0000","last_peered":"2024-09-11T01:22:56.058289+0000","last_clean":"2024-09-11T01:22:56.058289+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:56.058289+0000","last_undegraded":"2024-09-11T01:22:56.058289+0000","last_fullsized":"2024-09-11T01:22:56.058289+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12668928,"num_objects":3093,"num_object_clones":0,"num_object_copies":6186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17269,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17158","reported_seq":17280,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:53.240627+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:53.240627+0000","last_peered":"2024-09-11T01:22:53.240627+0000","last_clean":"2024-09-11T01:22:53.240627+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:53.240627+0000","last_undegraded":"2024-09-11T01:22:53.240627+0000","last_fullsized":"2024-09-11T01:22:53.240627+0000","mapping_epoch":31,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12775907,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13956,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17144","reported_seq":17411,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.213618+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:53.213618+0000","last_peered":"2024-09-11T01:22:53.213618+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:53.213618+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12546531,"num_objects":3064,"num_object_clones":0,"num_object_copies":6128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13632,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17621","reported_seq":17909,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.313065+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:53.313065+0000","last_peered":"2024-09-11T01:22:53.313065+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:53.313065+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13071302,"num_objects":3193,"num_object_clones":0,"num_object_copies":6386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3193,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14374,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1013},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17293","reported_seq":17412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:53.298011+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:53.298011+0000","last_peered":"2024-09-11T01:22:53.298011+0000","last_clean":"2024-09-11T01:22:53.298011+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:53.298011+0000","last_undegraded":"2024-09-11T01:22:53.298011+0000","last_fullsized":"2024-09-11T01:22:53.298011+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12824576,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14246,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17265","reported_seq":25609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.454525+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:56.454525+0000","last_peered":"2024-09-11T01:22:56.454525+0000","last_clean":"2024-09-11T01:22:56.454525+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:56.454525+0000","last_undegraded":"2024-09-11T01:22:56.454525+0000","last_fullsized":"2024-09-11T01:22:56.454525+0000","mapping_epoch":40,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222947,"num_objects":2985,"num_object_clones":0,"num_object_copies":5970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15132,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17393","reported_seq":17673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.250729+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:53.250729+0000","last_peered":"2024-09-11T01:22:53.250729+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:53.250729+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13111296,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13757,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-829}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17079","reported_seq":22714,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.027640+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:56.027640+0000","last_peered":"2024-09-11T01:22:56.027640+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:56.027640+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534243,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14848,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2769,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17530","reported_seq":17685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.048198+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:56.048198+0000","last_peered":"2024-09-11T01:22:56.048198+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:56.048198+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15167,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17318","reported_seq":25790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.414555+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:56.414555+0000","last_peered":"2024-09-11T01:22:56.414555+0000","last_clean":"2024-09-11T01:22:56.414555+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:56.414555+0000","last_undegraded":"2024-09-11T01:22:56.414555+0000","last_fullsized":"2024-09-11T01:22:56.414555+0000","mapping_epoch":42,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12427264,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17490","reported_seq":17775,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.300536+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:53.300536+0000","last_peered":"2024-09-11T01:22:53.300536+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:53.300536+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12926976,"num_objects":3156,"num_object_clones":0,"num_object_copies":6312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13883,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17419","reported_seq":17720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.256602+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:53.256602+0000","last_peered":"2024-09-11T01:22:53.256602+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:53.256602+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13029376,"num_objects":3181,"num_object_clones":0,"num_object_copies":6362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13954,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17041","reported_seq":17281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:53.231923+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:53.231923+0000","last_peered":"2024-09-11T01:22:53.231923+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:53.231923+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12686278,"num_objects":3099,"num_object_clones":0,"num_object_copies":6198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3099,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13698,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":203051766,"num_objects":49583,"num_object_clones":0,"num_object_copies":99166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49583,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":235658,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11635,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":606945280,"data_stored":606792616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547196,"kb_used_data":230892,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93824644,"statfs":{"total":96636764160,"available":96076435456,"internally_reserved":0,"allocated":236433408,"data_stored":236151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589408,"kb_used_data":146768,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93782432,"statfs":{"total":96636764160,"available":96033210368,"internally_reserved":0,"allocated":150290432,"data_stored":149968397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632904,"kb_used_data":218100,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93738936,"statfs":{"total":96636764160,"available":95988670464,"internally_reserved":0,"allocated":223334400,"data_stored":223052567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":424734378},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,5,0,1,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":221585408,"data_stored":221549278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149319680,"data_stored":149243348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236040192,"data_stored":235999990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:01.214 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:01.796 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:01.797 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:01.824 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-11T01:23:00.615408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197608201,"num_objects":48256,"num_object_clones":0,"num_object_copies":96512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48256,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":236989,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11636,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1752884,"kb_used_data":587456,"kb_used_omap":12,"kb_used_meta":1165363,"kb_avail":281362636,"statfs":{"total":289910292480,"available":288115339264,"internally_reserved":0,"allocated":601554944,"data_stored":600669248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1193331719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,21,17,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19251200,"num_objects":-4700,"num_object_clones":0,"num_object_copies":-9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4700,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004654"},"pg_stats":[{"pgid":"2.f","version":"43'17211","reported_seq":17435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.386356+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:58.386356+0000","last_peered":"2024-09-11T01:22:58.386356+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:58.386356+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12398592,"num_objects":3027,"num_object_clones":0,"num_object_copies":6054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17211,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17429","reported_seq":17510,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.049556+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:56.049556+0000","last_peered":"2024-09-11T01:22:56.049556+0000","last_clean":"2024-09-11T01:22:56.049556+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:56.049556+0000","last_undegraded":"2024-09-11T01:22:56.049556+0000","last_fullsized":"2024-09-11T01:22:56.049556+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15527,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17076","reported_seq":17245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.967518+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:55.967518+0000","last_peered":"2024-09-11T01:22:55.967518+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:55.967518+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161048,"num_objects":2970,"num_object_clones":0,"num_object_copies":5940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17269","reported_seq":17315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.058289+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:56.058289+0000","last_peered":"2024-09-11T01:22:56.058289+0000","last_clean":"2024-09-11T01:22:56.058289+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:56.058289+0000","last_undegraded":"2024-09-11T01:22:56.058289+0000","last_fullsized":"2024-09-11T01:22:56.058289+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12668928,"num_objects":3093,"num_object_clones":0,"num_object_copies":6186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17269,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17298","reported_seq":17420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:58.343718+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:58.343718+0000","last_peered":"2024-09-11T01:22:58.343718+0000","last_clean":"2024-09-11T01:22:58.343718+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:58.343718+0000","last_undegraded":"2024-09-11T01:22:58.343718+0000","last_fullsized":"2024-09-11T01:22:58.343718+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202467,"num_objects":2980,"num_object_clones":0,"num_object_copies":5960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14096,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17284","reported_seq":17551,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.356101+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:58.356101+0000","last_peered":"2024-09-11T01:22:58.356101+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:58.356101+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11973091,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13772,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1125},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17772","reported_seq":18060,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.358985+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:58.358985+0000","last_peered":"2024-09-11T01:22:58.358985+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:58.358985+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12452806,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14525,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1164},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17435","reported_seq":17556,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:58.345127+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:58.345127+0000","last_peered":"2024-09-11T01:22:58.345127+0000","last_clean":"2024-09-11T01:22:58.345127+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:58.345127+0000","last_undegraded":"2024-09-11T01:22:58.345127+0000","last_fullsized":"2024-09-11T01:22:58.345127+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12242944,"num_objects":2989,"num_object_clones":0,"num_object_copies":5978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14388,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17265","reported_seq":25609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.454525+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:56.454525+0000","last_peered":"2024-09-11T01:22:56.454525+0000","last_clean":"2024-09-11T01:22:56.454525+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:56.454525+0000","last_undegraded":"2024-09-11T01:22:56.454525+0000","last_fullsized":"2024-09-11T01:22:56.454525+0000","mapping_epoch":40,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222947,"num_objects":2985,"num_object_clones":0,"num_object_copies":5970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15132,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17557","reported_seq":17837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.275262+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:58.275262+0000","last_peered":"2024-09-11T01:22:58.275262+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:58.275262+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12439552,"num_objects":3037,"num_object_clones":0,"num_object_copies":6074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13921,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17079","reported_seq":22714,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.027640+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:56.027640+0000","last_peered":"2024-09-11T01:22:56.027640+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:56.027640+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534243,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14848,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2769,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17530","reported_seq":17685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.048198+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:56.048198+0000","last_peered":"2024-09-11T01:22:56.048198+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:56.048198+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15167,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17318","reported_seq":25790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.414555+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:56.414555+0000","last_peered":"2024-09-11T01:22:56.414555+0000","last_clean":"2024-09-11T01:22:56.414555+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:56.414555+0000","last_undegraded":"2024-09-11T01:22:56.414555+0000","last_fullsized":"2024-09-11T01:22:56.414555+0000","mapping_epoch":42,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12427264,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17653","reported_seq":17937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.321156+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:58.321156+0000","last_peered":"2024-09-11T01:22:58.321156+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:58.321156+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12259328,"num_objects":2993,"num_object_clones":0,"num_object_copies":5986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14046,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17570","reported_seq":17871,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.362542+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:58.362542+0000","last_peered":"2024-09-11T01:22:58.362542+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:58.362542+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12410880,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17178","reported_seq":17418,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.319675+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:58.319675+0000","last_peered":"2024-09-11T01:22:58.319675+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:58.319675+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12125126,"num_objects":2962,"num_object_clones":0,"num_object_copies":5924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13835,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1140},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":197608182,"num_objects":48254,"num_object_clones":0,"num_object_copies":96508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48254,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":236987,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11635,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":599797760,"data_stored":599645096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547196,"kb_used_data":230892,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93824644,"statfs":{"total":96636764160,"available":96076435456,"internally_reserved":0,"allocated":236433408,"data_stored":236151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589408,"kb_used_data":146768,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93782432,"statfs":{"total":96636764160,"available":96033210368,"internally_reserved":0,"allocated":150290432,"data_stored":149968397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616280,"kb_used_data":209796,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93755560,"statfs":{"total":96636764160,"available":96005693440,"internally_reserved":0,"allocated":214831104,"data_stored":214549271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,15,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214437888,"data_stored":214401758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149319680,"data_stored":149243348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236040192,"data_stored":235999990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:01.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:02.418 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:02.445 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-11T01:23:00.615408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":197608201,"num_objects":48256,"num_object_clones":0,"num_object_copies":96512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48256,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":236989,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11636,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1752884,"kb_used_data":587456,"kb_used_omap":12,"kb_used_meta":1165363,"kb_avail":281362636,"statfs":{"total":289910292480,"available":288115339264,"internally_reserved":0,"allocated":601554944,"data_stored":600669248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1193331719},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,21,17,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19251200,"num_objects":-4700,"num_object_clones":0,"num_object_copies":-9400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4700,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004654"},"pg_stats":[{"pgid":"2.f","version":"43'17211","reported_seq":17435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.386356+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:58.386356+0000","last_peered":"2024-09-11T01:22:58.386356+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:58.386356+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12398592,"num_objects":3027,"num_object_clones":0,"num_object_copies":6054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17211,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17429","reported_seq":17510,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.049556+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:22:56.049556+0000","last_peered":"2024-09-11T01:22:56.049556+0000","last_clean":"2024-09-11T01:22:56.049556+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:22:56.049556+0000","last_undegraded":"2024-09-11T01:22:56.049556+0000","last_fullsized":"2024-09-11T01:22:56.049556+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15527,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17076","reported_seq":17245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.967518+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:22:55.967518+0000","last_peered":"2024-09-11T01:22:55.967518+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:22:55.967518+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12161048,"num_objects":2970,"num_object_clones":0,"num_object_copies":5940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2970,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15137,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17269","reported_seq":17315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.058289+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:22:56.058289+0000","last_peered":"2024-09-11T01:22:56.058289+0000","last_clean":"2024-09-11T01:22:56.058289+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:22:56.058289+0000","last_undegraded":"2024-09-11T01:22:56.058289+0000","last_fullsized":"2024-09-11T01:22:56.058289+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12668928,"num_objects":3093,"num_object_clones":0,"num_object_copies":6186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17269,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17298","reported_seq":17420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:58.343718+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:58.343718+0000","last_peered":"2024-09-11T01:22:58.343718+0000","last_clean":"2024-09-11T01:22:58.343718+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:58.343718+0000","last_undegraded":"2024-09-11T01:22:58.343718+0000","last_fullsized":"2024-09-11T01:22:58.343718+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202467,"num_objects":2980,"num_object_clones":0,"num_object_copies":5960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14096,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17284","reported_seq":17551,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.356101+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:58.356101+0000","last_peered":"2024-09-11T01:22:58.356101+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:58.356101+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11973091,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13772,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1125},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17772","reported_seq":18060,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.358985+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:58.358985+0000","last_peered":"2024-09-11T01:22:58.358985+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:58.358985+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12452806,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14525,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1164},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17435","reported_seq":17556,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:58.345127+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:58.345127+0000","last_peered":"2024-09-11T01:22:58.345127+0000","last_clean":"2024-09-11T01:22:58.345127+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:58.345127+0000","last_undegraded":"2024-09-11T01:22:58.345127+0000","last_fullsized":"2024-09-11T01:22:58.345127+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12242944,"num_objects":2989,"num_object_clones":0,"num_object_copies":5978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14388,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17265","reported_seq":25609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.454525+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:22:56.454525+0000","last_peered":"2024-09-11T01:22:56.454525+0000","last_clean":"2024-09-11T01:22:56.454525+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:22:56.454525+0000","last_undegraded":"2024-09-11T01:22:56.454525+0000","last_fullsized":"2024-09-11T01:22:56.454525+0000","mapping_epoch":40,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12222947,"num_objects":2985,"num_object_clones":0,"num_object_copies":5970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15132,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17557","reported_seq":17837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.275262+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:58.275262+0000","last_peered":"2024-09-11T01:22:58.275262+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:58.275262+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12439552,"num_objects":3037,"num_object_clones":0,"num_object_copies":6074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13921,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17079","reported_seq":22714,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.027640+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:22:56.027640+0000","last_peered":"2024-09-11T01:22:56.027640+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:22:56.027640+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7000","ondisk_log_start":"39'7000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12534243,"num_objects":3061,"num_object_clones":0,"num_object_copies":6122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14848,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2769,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17530","reported_seq":17685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.048198+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:22:56.048198+0000","last_peered":"2024-09-11T01:22:56.048198+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:22:56.048198+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12419072,"num_objects":3032,"num_object_clones":0,"num_object_copies":6064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15167,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17318","reported_seq":25790,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.414555+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:22:56.414555+0000","last_peered":"2024-09-11T01:22:56.414555+0000","last_clean":"2024-09-11T01:22:56.414555+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:22:56.414555+0000","last_undegraded":"2024-09-11T01:22:56.414555+0000","last_fullsized":"2024-09-11T01:22:56.414555+0000","mapping_epoch":42,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12427264,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14008,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17653","reported_seq":17937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.321156+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:58.321156+0000","last_peered":"2024-09-11T01:22:58.321156+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:58.321156+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12259328,"num_objects":2993,"num_object_clones":0,"num_object_copies":5986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14046,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17570","reported_seq":17871,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.362542+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:58.362542+0000","last_peered":"2024-09-11T01:22:58.362542+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:58.362542+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12410880,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17178","reported_seq":17418,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.319675+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:58.319675+0000","last_peered":"2024-09-11T01:22:58.319675+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:58.319675+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12125126,"num_objects":2962,"num_object_clones":0,"num_object_copies":5924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13835,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1140},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":197608182,"num_objects":48254,"num_object_clones":0,"num_object_copies":96508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48254,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":236987,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11635,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":599797760,"data_stored":599645096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547196,"kb_used_data":230892,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93824644,"statfs":{"total":96636764160,"available":96076435456,"internally_reserved":0,"allocated":236433408,"data_stored":236151580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589408,"kb_used_data":146768,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93782432,"statfs":{"total":96636764160,"available":96033210368,"internally_reserved":0,"allocated":150290432,"data_stored":149968397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616280,"kb_used_data":209796,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93755560,"statfs":{"total":96636764160,"available":96005693440,"internally_reserved":0,"allocated":214831104,"data_stored":214549271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,15,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214437888,"data_stored":214401758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":149319680,"data_stored":149243348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":236040192,"data_stored":235999990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:02.446 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:02.446 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17211", 'reported_seq': 17435, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.386356+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:22:58.386356+0000', 'last_peered': '2024-09-11T01:22:58.386356+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:22:58.386356+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7200", 'ondisk_log_start': "39'7200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12398592, 'num_objects': 3027, 'num_object_clones': 0, 'num_object_copies': 6054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3027, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17211, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1021}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.446 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:02.446 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17076", 'reported_seq': 17245, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.967518+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:22:55.967518+0000', 'last_peered': '2024-09-11T01:22:55.967518+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:22:55.967518+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12161048, 'num_objects': 2970, 'num_object_clones': 0, 'num_object_copies': 5940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2970, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15137, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -993}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:02.446 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:02.446 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17284", 'reported_seq': 17551, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.356101+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:22:58.356101+0000', 'last_peered': '2024-09-11T01:22:58.356101+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:22:58.356101+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7200", 'ondisk_log_start': "39'7200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11973091, 'num_objects': 2924, 'num_object_clones': 0, 'num_object_copies': 5848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2924, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13772, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1125}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'17772", 'reported_seq': 18060, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.358985+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:22:58.358985+0000', 'last_peered': '2024-09-11T01:22:58.358985+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:22:58.358985+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'7700", 'ondisk_log_start': "39'7700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12452806, 'num_objects': 3042, 'num_object_clones': 0, 'num_object_copies': 6084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3042, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14525, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1164}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'17557", 'reported_seq': 17837, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.275262+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:22:58.275262+0000', 'last_peered': '2024-09-11T01:22:58.275262+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:22:58.275262+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12439552, 'num_objects': 3037, 'num_object_clones': 0, 'num_object_copies': 6074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3037, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13921, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -993}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17079", 'reported_seq': 22714, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.027640+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:22:56.027640+0000', 'last_peered': '2024-09-11T01:22:56.027640+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:22:56.027640+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7000", 'ondisk_log_start': "39'7000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12534243, 'num_objects': 3061, 'num_object_clones': 0, 'num_object_copies': 6122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1259, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3061, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14848, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2769, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1802}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'17530", 'reported_seq': 17685, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.048198+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:22:56.048198+0000', 'last_peered': '2024-09-11T01:22:56.048198+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:22:56.048198+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12419072, 'num_objects': 3032, 'num_object_clones': 0, 'num_object_copies': 6064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3032, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15167, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1115}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:02.447 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17653", 'reported_seq': 17937, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.321156+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:22:58.321156+0000', 'last_peered': '2024-09-11T01:22:58.321156+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:22:58.321156+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7600", 'ondisk_log_start': "39'7600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12259328, 'num_objects': 2993, 'num_object_clones': 0, 'num_object_copies': 5986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14046, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1124}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.448 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:02.448 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'17570", 'reported_seq': 17871, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.362542+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:22:58.362542+0000', 'last_peered': '2024-09-11T01:22:58.362542+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:22:58.362542+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12410880, 'num_objects': 3030, 'num_object_clones': 0, 'num_object_copies': 6060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3030, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14105, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.448 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:02.448 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17178", 'reported_seq': 17418, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.319675+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:22:58.319675+0000', 'last_peered': '2024-09-11T01:22:58.319675+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:22:58.319675+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7100", 'ondisk_log_start': "39'7100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12125126, 'num_objects': 2962, 'num_object_clones': 0, 'num_object_copies': 5924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2962, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13835, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1140}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:02.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:03.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:03.165 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37880/94476 objects degraded (40.095%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":583,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":47238,"data_bytes":193438473,"bytes_used":1795825664,"bytes_avail":288114466816,"bytes_total":289910292480,"degraded_objects":37880,"degraded_total":94476,"degraded_ratio":0.40094838900884883,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":328,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:03.191 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:03.191 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:03.782 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:03.784 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:03.817 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2024-09-11T01:23:02.615846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":193438473,"num_objects":47238,"num_object_clones":0,"num_object_copies":94476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47238,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":238007,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11755,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1753736,"kb_used_data":575816,"kb_used_omap":12,"kb_used_meta":1177843,"kb_avail":281361784,"statfs":{"total":289910292480,"available":288114466816,"internally_reserved":0,"allocated":589635584,"data_stored":588749888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1206111239},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,6,24,13,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23420928,"num_objects":-5718,"num_object_clones":0,"num_object_copies":-11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-328,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5718,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004275"},"pg_stats":[{"pgid":"2.f","version":"43'17211","reported_seq":17435,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.386356+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:22:58.386356+0000","last_peered":"2024-09-11T01:22:58.386356+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:22:58.386356+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12398592,"num_objects":3027,"num_object_clones":0,"num_object_copies":6054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3027,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17211,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1021}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17571","reported_seq":17652,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:01.067061+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:01.067061+0000","last_peered":"2024-09-11T01:23:01.067061+0000","last_clean":"2024-09-11T01:23:01.067061+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:01.067061+0000","last_undegraded":"2024-09-11T01:23:01.067061+0000","last_fullsized":"2024-09-11T01:23:01.067061+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12088262,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15669,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17208","reported_seq":17377,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:01.067559+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:01.067559+0000","last_peered":"2024-09-11T01:23:01.067559+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:01.067559+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620376,"num_objects":2838,"num_object_clones":0,"num_object_copies":5676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15269,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17424","reported_seq":17470,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:01.091138+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:01.091138+0000","last_peered":"2024-09-11T01:23:01.091138+0000","last_clean":"2024-09-11T01:23:01.091138+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:01.091138+0000","last_undegraded":"2024-09-11T01:23:01.091138+0000","last_fullsized":"2024-09-11T01:23:01.091138+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12034048,"num_objects":2938,"num_object_clones":0,"num_object_copies":5876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17424,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17298","reported_seq":17420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:58.343718+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:22:58.343718+0000","last_peered":"2024-09-11T01:22:58.343718+0000","last_clean":"2024-09-11T01:22:58.343718+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:22:58.343718+0000","last_undegraded":"2024-09-11T01:22:58.343718+0000","last_fullsized":"2024-09-11T01:22:58.343718+0000","mapping_epoch":31,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12202467,"num_objects":2980,"num_object_clones":0,"num_object_copies":5960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2980,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14096,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17284","reported_seq":17551,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.356101+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:22:58.356101+0000","last_peered":"2024-09-11T01:22:58.356101+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:22:58.356101+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11973091,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13772,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1125},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17772","reported_seq":18060,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.358985+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:22:58.358985+0000","last_peered":"2024-09-11T01:22:58.358985+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:22:58.358985+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12452806,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14525,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1164},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17435","reported_seq":17556,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:22:58.345127+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:22:58.345127+0000","last_peered":"2024-09-11T01:22:58.345127+0000","last_clean":"2024-09-11T01:22:58.345127+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:22:58.345127+0000","last_undegraded":"2024-09-11T01:22:58.345127+0000","last_fullsized":"2024-09-11T01:22:58.345127+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12242944,"num_objects":2989,"num_object_clones":0,"num_object_copies":5978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14388,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17415","reported_seq":25759,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:01.401719+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:01.401719+0000","last_peered":"2024-09-11T01:23:01.401719+0000","last_clean":"2024-09-11T01:23:01.401719+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:01.401719+0000","last_undegraded":"2024-09-11T01:23:01.401719+0000","last_fullsized":"2024-09-11T01:23:01.401719+0000","mapping_epoch":40,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11608547,"num_objects":2835,"num_object_clones":0,"num_object_copies":5670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2835,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15282,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17557","reported_seq":17837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.275262+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:22:58.275262+0000","last_peered":"2024-09-11T01:22:58.275262+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:22:58.275262+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12439552,"num_objects":3037,"num_object_clones":0,"num_object_copies":6074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13921,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17216","reported_seq":23088,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:01.067723+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:01.067723+0000","last_peered":"2024-09-11T01:23:01.067723+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:01.067723+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7200","ondisk_log_start":"39'7200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11973091,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1140,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14985,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2888,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1784}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17681","reported_seq":17836,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:00.945828+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:00.945828+0000","last_peered":"2024-09-11T01:23:00.945828+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:00.945828+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11800576,"num_objects":2881,"num_object_clones":0,"num_object_copies":5762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15318,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1266}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17469","reported_seq":25940,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:01.475728+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:01.475728+0000","last_peered":"2024-09-11T01:23:01.475728+0000","last_clean":"2024-09-11T01:23:01.475728+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:01.475728+0000","last_undegraded":"2024-09-11T01:23:01.475728+0000","last_fullsized":"2024-09-11T01:23:01.475728+0000","mapping_epoch":42,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14159,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17653","reported_seq":17937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.321156+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:22:58.321156+0000","last_peered":"2024-09-11T01:22:58.321156+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:22:58.321156+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12259328,"num_objects":2993,"num_object_clones":0,"num_object_copies":5986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14046,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1124}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17570","reported_seq":17871,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.362542+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:22:58.362542+0000","last_peered":"2024-09-11T01:22:58.362542+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:22:58.362542+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12410880,"num_objects":3030,"num_object_clones":0,"num_object_copies":6060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1035}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17178","reported_seq":17418,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.319675+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:22:58.319675+0000","last_peered":"2024-09-11T01:22:58.319675+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:22:58.319675+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7100","ondisk_log_start":"39'7100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12125126,"num_objects":2962,"num_object_clones":0,"num_object_copies":5924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13835,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1140},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":193438454,"num_objects":47236,"num_object_clones":0,"num_object_copies":94472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37880,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47236,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":238005,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11754,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":587833344,"data_stored":587680680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546848,"kb_used_data":226372,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93824992,"statfs":{"total":96636764160,"available":96076791808,"internally_reserved":0,"allocated":231804928,"data_stored":231523100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790045,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590608,"kb_used_data":139648,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93781232,"statfs":{"total":96636764160,"available":96031981568,"internally_reserved":0,"allocated":142999552,"data_stored":142677517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616280,"kb_used_data":209796,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93755560,"statfs":{"total":96636764160,"available":96005693440,"internally_reserved":0,"allocated":214831104,"data_stored":214549271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":416214698},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,15,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214437888,"data_stored":214401758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":142061568,"data_stored":141985236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":231333888,"data_stored":231293686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:06.820 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:07.389 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:07.392 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:07.413 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-11T01:23:06.617195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184001289,"num_objects":44934,"num_object_clones":0,"num_object_copies":89868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44934,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":240311,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11874,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1748900,"kb_used_data":558492,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281366620,"statfs":{"total":289910292480,"available":288119418880,"internally_reserved":0,"allocated":571895808,"data_stored":571010112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1218890759},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,8,10,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23367680,"num_objects":-5705,"num_object_clones":0,"num_object_copies":-11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003843"},"pg_stats":[{"pgid":"2.f","version":"43'17370","reported_seq":17594,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.331993+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:03.331993+0000","last_peered":"2024-09-11T01:23:03.331993+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:03.331993+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11747328,"num_objects":2868,"num_object_clones":0,"num_object_copies":5736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17370,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17710","reported_seq":17791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:05.977752+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:05.977752+0000","last_peered":"2024-09-11T01:23:05.977752+0000","last_clean":"2024-09-11T01:23:05.977752+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:05.977752+0000","last_undegraded":"2024-09-11T01:23:05.977752+0000","last_fullsized":"2024-09-11T01:23:05.977752+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11518918,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15808,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17359","reported_seq":17528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.022486+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:06.022486+0000","last_peered":"2024-09-11T01:23:06.022486+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:06.022486+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11001880,"num_objects":2687,"num_object_clones":0,"num_object_copies":5374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15420,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17562","reported_seq":17608,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.000111+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:06.000111+0000","last_peered":"2024-09-11T01:23:06.000111+0000","last_clean":"2024-09-11T01:23:06.000111+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:06.000111+0000","last_undegraded":"2024-09-11T01:23:06.000111+0000","last_fullsized":"2024-09-11T01:23:06.000111+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11468800,"num_objects":2800,"num_object_clones":0,"num_object_copies":5600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17562,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17423","reported_seq":17546,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:03.388528+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:03.388528+0000","last_peered":"2024-09-11T01:23:03.388528+0000","last_clean":"2024-09-11T01:23:03.388528+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:03.388528+0000","last_undegraded":"2024-09-11T01:23:03.388528+0000","last_fullsized":"2024-09-11T01:23:03.388528+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11690467,"num_objects":2855,"num_object_clones":0,"num_object_copies":5710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14221,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17429","reported_seq":17695,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.395929+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:03.395929+0000","last_peered":"2024-09-11T01:23:03.395929+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:03.395929+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11379171,"num_objects":2779,"num_object_clones":0,"num_object_copies":5558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13917,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1270},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17920","reported_seq":18204,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.402461+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:03.402461+0000","last_peered":"2024-09-11T01:23:03.402461+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:03.402461+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11846598,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14673,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1312},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17581","reported_seq":17702,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:03.320792+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:03.320792+0000","last_peered":"2024-09-11T01:23:03.320792+0000","last_clean":"2024-09-11T01:23:03.320792+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:03.320792+0000","last_undegraded":"2024-09-11T01:23:03.320792+0000","last_fullsized":"2024-09-11T01:23:03.320792+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11644928,"num_objects":2843,"num_object_clones":0,"num_object_copies":5686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17558","reported_seq":25902,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.490806+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:06.490806+0000","last_peered":"2024-09-11T01:23:06.490806+0000","last_clean":"2024-09-11T01:23:06.490806+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:06.490806+0000","last_undegraded":"2024-09-11T01:23:06.490806+0000","last_fullsized":"2024-09-11T01:23:06.490806+0000","mapping_epoch":40,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022819,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15425,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17692","reported_seq":17972,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.325845+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:03.325845+0000","last_peered":"2024-09-11T01:23:03.325845+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:03.325845+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14056,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1128}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17355","reported_seq":23466,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.086026+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:06.086026+0000","last_peered":"2024-09-11T01:23:06.086026+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:06.086026+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11403747,"num_objects":2785,"num_object_clones":0,"num_object_copies":5570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15124,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1764}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17820","reported_seq":17975,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.050086+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:06.050086+0000","last_peered":"2024-09-11T01:23:06.050086+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:06.050086+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11231232,"num_objects":2742,"num_object_clones":0,"num_object_copies":5484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15457,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17619","reported_seq":26091,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.482549+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:06.482549+0000","last_peered":"2024-09-11T01:23:06.482549+0000","last_clean":"2024-09-11T01:23:06.482549+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:06.482549+0000","last_undegraded":"2024-09-11T01:23:06.482549+0000","last_fullsized":"2024-09-11T01:23:06.482549+0000","mapping_epoch":42,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11194368,"num_objects":2733,"num_object_clones":0,"num_object_copies":5466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17820","reported_seq":18106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.350132+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:03.350132+0000","last_peered":"2024-09-11T01:23:03.350132+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:03.350132+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575296,"num_objects":2826,"num_object_clones":0,"num_object_copies":5652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14213,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17698","reported_seq":17996,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.384016+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:03.384016+0000","last_peered":"2024-09-11T01:23:03.384016+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:03.384016+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14233,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17330","reported_seq":17570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.340243+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:03.340243+0000","last_peered":"2024-09-11T01:23:03.340243+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:03.340243+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11502534,"num_objects":2810,"num_object_clones":0,"num_object_copies":5620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2810,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13987,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1292},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184001270,"num_objects":44932,"num_object_clones":0,"num_object_copies":89864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44932,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":240309,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11873,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568860672,"data_stored":568708008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546576,"kb_used_data":221948,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93825264,"statfs":{"total":96636764160,"available":96077070336,"internally_reserved":0,"allocated":227274752,"data_stored":226992924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790046,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587684,"kb_used_data":132556,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93784156,"statfs":{"total":96636764160,"available":96034975744,"internally_reserved":0,"allocated":135737344,"data_stored":135415309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614640,"kb_used_data":203988,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93757200,"statfs":{"total":96636764160,"available":96007372800,"internally_reserved":0,"allocated":208883712,"data_stored":208601879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207282176,"data_stored":207246046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":134758400,"data_stored":134682068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":226820096,"data_stored":226779894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:07.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:07.985 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:07.986 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:08.013 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-11T01:23:06.617195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184001289,"num_objects":44934,"num_object_clones":0,"num_object_copies":89868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44934,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":240311,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11874,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1748900,"kb_used_data":558492,"kb_used_omap":12,"kb_used_meta":1190323,"kb_avail":281366620,"statfs":{"total":289910292480,"available":288119418880,"internally_reserved":0,"allocated":571895808,"data_stored":571010112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1218890759},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,8,10,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23367680,"num_objects":-5705,"num_object_clones":0,"num_object_copies":-11410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5705,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003843"},"pg_stats":[{"pgid":"2.f","version":"43'17370","reported_seq":17594,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.331993+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:03.331993+0000","last_peered":"2024-09-11T01:23:03.331993+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:03.331993+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11747328,"num_objects":2868,"num_object_clones":0,"num_object_copies":5736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17370,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17710","reported_seq":17791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:05.977752+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:05.977752+0000","last_peered":"2024-09-11T01:23:05.977752+0000","last_clean":"2024-09-11T01:23:05.977752+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:05.977752+0000","last_undegraded":"2024-09-11T01:23:05.977752+0000","last_fullsized":"2024-09-11T01:23:05.977752+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11518918,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15808,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17359","reported_seq":17528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.022486+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:06.022486+0000","last_peered":"2024-09-11T01:23:06.022486+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:06.022486+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11001880,"num_objects":2687,"num_object_clones":0,"num_object_copies":5374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15420,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17562","reported_seq":17608,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.000111+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:06.000111+0000","last_peered":"2024-09-11T01:23:06.000111+0000","last_clean":"2024-09-11T01:23:06.000111+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:06.000111+0000","last_undegraded":"2024-09-11T01:23:06.000111+0000","last_fullsized":"2024-09-11T01:23:06.000111+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11468800,"num_objects":2800,"num_object_clones":0,"num_object_copies":5600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17562,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17423","reported_seq":17546,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:03.388528+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:03.388528+0000","last_peered":"2024-09-11T01:23:03.388528+0000","last_clean":"2024-09-11T01:23:03.388528+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:03.388528+0000","last_undegraded":"2024-09-11T01:23:03.388528+0000","last_fullsized":"2024-09-11T01:23:03.388528+0000","mapping_epoch":31,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11690467,"num_objects":2855,"num_object_clones":0,"num_object_copies":5710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14221,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17429","reported_seq":17695,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.395929+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:03.395929+0000","last_peered":"2024-09-11T01:23:03.395929+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:03.395929+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11379171,"num_objects":2779,"num_object_clones":0,"num_object_copies":5558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13917,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1270},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'17920","reported_seq":18204,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.402461+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:03.402461+0000","last_peered":"2024-09-11T01:23:03.402461+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:03.402461+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11846598,"num_objects":2894,"num_object_clones":0,"num_object_copies":5788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2894,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14673,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1312},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17581","reported_seq":17702,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:03.320792+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:03.320792+0000","last_peered":"2024-09-11T01:23:03.320792+0000","last_clean":"2024-09-11T01:23:03.320792+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:03.320792+0000","last_undegraded":"2024-09-11T01:23:03.320792+0000","last_fullsized":"2024-09-11T01:23:03.320792+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11644928,"num_objects":2843,"num_object_clones":0,"num_object_copies":5686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17558","reported_seq":25902,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.490806+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:06.490806+0000","last_peered":"2024-09-11T01:23:06.490806+0000","last_clean":"2024-09-11T01:23:06.490806+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:06.490806+0000","last_undegraded":"2024-09-11T01:23:06.490806+0000","last_fullsized":"2024-09-11T01:23:06.490806+0000","mapping_epoch":40,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022819,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15425,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17692","reported_seq":17972,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.325845+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:03.325845+0000","last_peered":"2024-09-11T01:23:03.325845+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:03.325845+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14056,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1128}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17355","reported_seq":23466,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.086026+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:06.086026+0000","last_peered":"2024-09-11T01:23:06.086026+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:06.086026+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11403747,"num_objects":2785,"num_object_clones":0,"num_object_copies":5570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15124,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1764}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17820","reported_seq":17975,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.050086+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:06.050086+0000","last_peered":"2024-09-11T01:23:06.050086+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:06.050086+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11231232,"num_objects":2742,"num_object_clones":0,"num_object_copies":5484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15457,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17619","reported_seq":26091,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.482549+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:06.482549+0000","last_peered":"2024-09-11T01:23:06.482549+0000","last_clean":"2024-09-11T01:23:06.482549+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:06.482549+0000","last_undegraded":"2024-09-11T01:23:06.482549+0000","last_fullsized":"2024-09-11T01:23:06.482549+0000","mapping_epoch":42,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11194368,"num_objects":2733,"num_object_clones":0,"num_object_copies":5466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17820","reported_seq":18106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.350132+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:03.350132+0000","last_peered":"2024-09-11T01:23:03.350132+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:03.350132+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11575296,"num_objects":2826,"num_object_clones":0,"num_object_copies":5652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14213,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17698","reported_seq":17996,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.384016+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:03.384016+0000","last_peered":"2024-09-11T01:23:03.384016+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:03.384016+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11886592,"num_objects":2902,"num_object_clones":0,"num_object_copies":5804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14233,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1163}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17330","reported_seq":17570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.340243+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:03.340243+0000","last_peered":"2024-09-11T01:23:03.340243+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:03.340243+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11502534,"num_objects":2810,"num_object_clones":0,"num_object_copies":5620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2810,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13987,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1292},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184001270,"num_objects":44932,"num_object_clones":0,"num_object_copies":89864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44932,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":240309,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11873,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":568860672,"data_stored":568708008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546576,"kb_used_data":221948,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93825264,"statfs":{"total":96636764160,"available":96077070336,"internally_reserved":0,"allocated":227274752,"data_stored":226992924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790046,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587684,"kb_used_data":132556,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93784156,"statfs":{"total":96636764160,"available":96034975744,"internally_reserved":0,"allocated":135737344,"data_stored":135415309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614640,"kb_used_data":203988,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93757200,"statfs":{"total":96636764160,"available":96007372800,"internally_reserved":0,"allocated":208883712,"data_stored":208601879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":420474538},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207282176,"data_stored":207246046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":134758400,"data_stored":134682068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":226820096,"data_stored":226779894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:08.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:08.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17370", 'reported_seq': 17594, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.331993+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:03.331993+0000', 'last_peered': '2024-09-11T01:23:03.331993+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:03.331993+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11747328, 'num_objects': 2868, 'num_object_clones': 0, 'num_object_copies': 5736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17370, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1180}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:08.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17359", 'reported_seq': 17528, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:06.022486+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:06.022486+0000', 'last_peered': '2024-09-11T01:23:06.022486+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:06.022486+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11001880, 'num_objects': 2687, 'num_object_clones': 0, 'num_object_copies': 5374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2687, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15420, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:08.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:08.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17429", 'reported_seq': 17695, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.395929+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:03.395929+0000', 'last_peered': '2024-09-11T01:23:03.395929+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:03.395929+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7400", 'ondisk_log_start': "39'7400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11379171, 'num_objects': 2779, 'num_object_clones': 0, 'num_object_copies': 5558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2779, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13917, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1270}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'17920", 'reported_seq': 18204, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.402461+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:03.402461+0000', 'last_peered': '2024-09-11T01:23:03.402461+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:03.402461+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11846598, 'num_objects': 2894, 'num_object_clones': 0, 'num_object_copies': 5788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2894, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14673, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1312}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'17692", 'reported_seq': 17972, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.325845+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:03.325845+0000', 'last_peered': '2024-09-11T01:23:03.325845+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:03.325845+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'7600", 'ondisk_log_start': "39'7600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11886592, 'num_objects': 2902, 'num_object_clones': 0, 'num_object_copies': 5804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14056, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1128}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17355", 'reported_seq': 23466, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:06.086026+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:06.086026+0000', 'last_peered': '2024-09-11T01:23:06.086026+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:06.086026+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11403747, 'num_objects': 2785, 'num_object_clones': 0, 'num_object_copies': 5570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1021, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2785, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15124, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3007, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1764}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'17820", 'reported_seq': 17975, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:06.050086+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:06.050086+0000', 'last_peered': '2024-09-11T01:23:06.050086+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:06.050086+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11231232, 'num_objects': 2742, 'num_object_clones': 0, 'num_object_copies': 5484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15457, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1405}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:08.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17820", 'reported_seq': 18106, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.350132+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:03.350132+0000', 'last_peered': '2024-09-11T01:23:03.350132+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:03.350132+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11575296, 'num_objects': 2826, 'num_object_clones': 0, 'num_object_copies': 5652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14213, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:08.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'17698", 'reported_seq': 17996, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.384016+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:03.384016+0000', 'last_peered': '2024-09-11T01:23:03.384016+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:03.384016+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'7600", 'ondisk_log_start': "39'7600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11886592, 'num_objects': 2902, 'num_object_clones': 0, 'num_object_copies': 5804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14233, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.016 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:08.016 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17330", 'reported_seq': 17570, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.340243+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:03.340243+0000', 'last_peered': '2024-09-11T01:23:03.340243+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:03.340243+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7300", 'ondisk_log_start': "39'7300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11502534, 'num_objects': 2810, 'num_object_clones': 0, 'num_object_copies': 5620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2810, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13987, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1292}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:08.016 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:08.740 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:08.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37761/89868 objects degraded (42.018%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":589,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":44934,"data_bytes":184001289,"bytes_used":1790873600,"bytes_avail":288119418880,"bytes_total":289910292480,"degraded_objects":37761,"degraded_total":89868,"degraded_ratio":0.42018293497129122,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":341,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":475},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:08.780 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:08.780 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:09.383 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:09.383 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:09.407 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-11T01:23:08.618036+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":178619145,"num_objects":43620,"num_object_clones":0,"num_object_copies":87240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43620,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":241625,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11874,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768232,"kb_used_data":551460,"kb_used_omap":12,"kb_used_meta":1216691,"kb_avail":281347288,"statfs":{"total":289910292480,"available":288099622912,"internally_reserved":0,"allocated":564695040,"data_stored":563809344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1245891591},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,6,13,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24432640,"num_objects":-5965,"num_object_clones":0,"num_object_copies":-11930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5965,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003811"},"pg_stats":[{"pgid":"2.f","version":"43'17534","reported_seq":17758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.332928+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:08.332928+0000","last_peered":"2024-09-11T01:23:08.332928+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:08.332928+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17534,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17710","reported_seq":17791,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:05.977752+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:05.977752+0000","last_peered":"2024-09-11T01:23:05.977752+0000","last_clean":"2024-09-11T01:23:05.977752+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:05.977752+0000","last_undegraded":"2024-09-11T01:23:05.977752+0000","last_fullsized":"2024-09-11T01:23:05.977752+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11518918,"num_objects":2814,"num_object_clones":0,"num_object_copies":5628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15808,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17359","reported_seq":17528,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.022486+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:06.022486+0000","last_peered":"2024-09-11T01:23:06.022486+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:06.022486+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11001880,"num_objects":2687,"num_object_clones":0,"num_object_copies":5374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15420,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17562","reported_seq":17608,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.000111+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:06.000111+0000","last_peered":"2024-09-11T01:23:06.000111+0000","last_clean":"2024-09-11T01:23:06.000111+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:06.000111+0000","last_undegraded":"2024-09-11T01:23:06.000111+0000","last_fullsized":"2024-09-11T01:23:06.000111+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11468800,"num_objects":2800,"num_object_clones":0,"num_object_copies":5600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17562,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17548","reported_seq":17671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:08.337812+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:08.337812+0000","last_peered":"2024-09-11T01:23:08.337812+0000","last_clean":"2024-09-11T01:23:08.337812+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:08.337812+0000","last_undegraded":"2024-09-11T01:23:08.337812+0000","last_fullsized":"2024-09-11T01:23:08.337812+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11178467,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14346,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17575","reported_seq":17839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.428582+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:08.428582+0000","last_peered":"2024-09-11T01:23:08.428582+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:08.428582+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10781155,"num_objects":2633,"num_object_clones":0,"num_object_copies":5266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14063,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1416},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18049","reported_seq":18338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.376615+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:08.376615+0000","last_peered":"2024-09-11T01:23:08.376615+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:08.376615+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11318214,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14802,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1441},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17704","reported_seq":17825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:08.309315+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:08.309315+0000","last_peered":"2024-09-11T01:23:08.309315+0000","last_clean":"2024-09-11T01:23:08.309315+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:08.309315+0000","last_undegraded":"2024-09-11T01:23:08.309315+0000","last_fullsized":"2024-09-11T01:23:08.309315+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11141120,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17558","reported_seq":25902,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.490806+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:06.490806+0000","last_peered":"2024-09-11T01:23:06.490806+0000","last_clean":"2024-09-11T01:23:06.490806+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:06.490806+0000","last_undegraded":"2024-09-11T01:23:06.490806+0000","last_fullsized":"2024-09-11T01:23:06.490806+0000","mapping_epoch":40,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11022819,"num_objects":2692,"num_object_clones":0,"num_object_copies":5384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2692,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15425,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17847","reported_seq":18125,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.393581+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:08.393581+0000","last_peered":"2024-09-11T01:23:08.393581+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:08.393581+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11251712,"num_objects":2747,"num_object_clones":0,"num_object_copies":5494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1283}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17355","reported_seq":23466,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.086026+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:06.086026+0000","last_peered":"2024-09-11T01:23:06.086026+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:06.086026+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7300","ondisk_log_start":"39'7300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11403747,"num_objects":2785,"num_object_clones":0,"num_object_copies":5570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1021,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15124,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3007,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1764}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17820","reported_seq":17975,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:06.050086+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:06.050086+0000","last_peered":"2024-09-11T01:23:06.050086+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:06.050086+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11231232,"num_objects":2742,"num_object_clones":0,"num_object_copies":5484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15457,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1405}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17619","reported_seq":26091,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:06.482549+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:06.482549+0000","last_peered":"2024-09-11T01:23:06.482549+0000","last_clean":"2024-09-11T01:23:06.482549+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:06.482549+0000","last_undegraded":"2024-09-11T01:23:06.482549+0000","last_fullsized":"2024-09-11T01:23:06.482549+0000","mapping_epoch":42,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11194368,"num_objects":2733,"num_object_clones":0,"num_object_copies":5466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14309,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17980","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.459771+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:08.459771+0000","last_peered":"2024-09-11T01:23:08.459771+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:08.459771+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10919936,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14373,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17860","reported_seq":18161,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.420189+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:08.420189+0000","last_peered":"2024-09-11T01:23:08.420189+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:08.420189+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11223040,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14395,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17480","reported_seq":17720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.362684+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:08.362684+0000","last_peered":"2024-09-11T01:23:08.362684+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:08.362684+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10888134,"num_objects":2660,"num_object_clones":0,"num_object_copies":5320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14137,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1442},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":178619126,"num_objects":43618,"num_object_clones":0,"num_object_copies":87236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43618,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":241623,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11873,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561676288,"data_stored":561523624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546576,"kb_used_data":221948,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93825264,"statfs":{"total":96636764160,"available":96077070336,"internally_reserved":0,"allocated":227274752,"data_stored":226992924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790046,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587684,"kb_used_data":132556,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93784156,"statfs":{"total":96636764160,"available":96034975744,"internally_reserved":0,"allocated":135737344,"data_stored":135415309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633972,"kb_used_data":196956,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93737868,"statfs":{"total":96636764160,"available":95987576832,"internally_reserved":0,"allocated":201682944,"data_stored":201401111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200097792,"data_stored":200061662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":134758400,"data_stored":134682068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":226820096,"data_stored":226779894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:12.410 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:12.997 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:13.000 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:13.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-11T01:23:12.619468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174514953,"num_objects":42618,"num_object_clones":0,"num_object_copies":85236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42618,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":242627,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11985,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1753472,"kb_used_data":540856,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281362048,"statfs":{"total":289910292480,"available":288114737152,"internally_reserved":0,"allocated":553836544,"data_stored":552950848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1241631751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,8,13,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23093248,"num_objects":-5638,"num_object_clones":0,"num_object_copies":-11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004060"},"pg_stats":[{"pgid":"2.f","version":"43'17534","reported_seq":17758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.332928+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:08.332928+0000","last_peered":"2024-09-11T01:23:08.332928+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:08.332928+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17534,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17853","reported_seq":17933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.067946+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:11.067946+0000","last_peered":"2024-09-11T01:23:11.067946+0000","last_clean":"2024-09-11T01:23:11.067946+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:11.067946+0000","last_undegraded":"2024-09-11T01:23:11.067946+0000","last_fullsized":"2024-09-11T01:23:11.067946+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10933190,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15951,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17503","reported_seq":17672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.001750+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:11.001750+0000","last_peered":"2024-09-11T01:23:11.001750+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:11.001750+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10412056,"num_objects":2543,"num_object_clones":0,"num_object_copies":5086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15564,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17708","reported_seq":17754,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:10.996775+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:10.996775+0000","last_peered":"2024-09-11T01:23:10.996775+0000","last_clean":"2024-09-11T01:23:10.996775+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:10.996775+0000","last_undegraded":"2024-09-11T01:23:10.996775+0000","last_fullsized":"2024-09-11T01:23:10.996775+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10870784,"num_objects":2654,"num_object_clones":0,"num_object_copies":5308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17708,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17548","reported_seq":17671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:08.337812+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:08.337812+0000","last_peered":"2024-09-11T01:23:08.337812+0000","last_clean":"2024-09-11T01:23:08.337812+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:08.337812+0000","last_undegraded":"2024-09-11T01:23:08.337812+0000","last_fullsized":"2024-09-11T01:23:08.337812+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11178467,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14346,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17575","reported_seq":17839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.428582+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:08.428582+0000","last_peered":"2024-09-11T01:23:08.428582+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:08.428582+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10781155,"num_objects":2633,"num_object_clones":0,"num_object_copies":5266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14063,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1416},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18049","reported_seq":18338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.376615+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:08.376615+0000","last_peered":"2024-09-11T01:23:08.376615+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:08.376615+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11318214,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14802,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1441},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17704","reported_seq":17825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:08.309315+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:08.309315+0000","last_peered":"2024-09-11T01:23:08.309315+0000","last_clean":"2024-09-11T01:23:08.309315+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:08.309315+0000","last_undegraded":"2024-09-11T01:23:08.309315+0000","last_fullsized":"2024-09-11T01:23:08.309315+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11141120,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17694","reported_seq":26038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.552912+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:11.552912+0000","last_peered":"2024-09-11T01:23:11.552912+0000","last_clean":"2024-09-11T01:23:11.552912+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:11.552912+0000","last_undegraded":"2024-09-11T01:23:11.552912+0000","last_fullsized":"2024-09-11T01:23:11.552912+0000","mapping_epoch":40,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10465763,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15561,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17847","reported_seq":18125,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.393581+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:08.393581+0000","last_peered":"2024-09-11T01:23:08.393581+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:08.393581+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11251712,"num_objects":2747,"num_object_clones":0,"num_object_copies":5494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1283}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17498","reported_seq":23830,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.101453+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:11.101453+0000","last_peered":"2024-09-11T01:23:11.101453+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:11.101453+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818019,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15267,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3118,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1732}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17974","reported_seq":18129,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.074755+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:11.074755+0000","last_peered":"2024-09-11T01:23:11.074755+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:11.074755+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10600448,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15611,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1559}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17755","reported_seq":26227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.538997+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:11.538997+0000","last_peered":"2024-09-11T01:23:11.538997+0000","last_clean":"2024-09-11T01:23:11.538997+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:11.538997+0000","last_undegraded":"2024-09-11T01:23:11.538997+0000","last_fullsized":"2024-09-11T01:23:11.538997+0000","mapping_epoch":42,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10637312,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17980","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.459771+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:08.459771+0000","last_peered":"2024-09-11T01:23:08.459771+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:08.459771+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10919936,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14373,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17860","reported_seq":18161,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.420189+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:08.420189+0000","last_peered":"2024-09-11T01:23:08.420189+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:08.420189+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11223040,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14395,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17480","reported_seq":17720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.362684+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:08.362684+0000","last_peered":"2024-09-11T01:23:08.362684+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:08.362684+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10888134,"num_objects":2660,"num_object_clones":0,"num_object_copies":5320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14137,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1442},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":174514934,"num_objects":42616,"num_object_clones":0,"num_object_copies":85232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42616,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":242625,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11984,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":549867520,"data_stored":549714856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547116,"kb_used_data":218332,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93824724,"statfs":{"total":96636764160,"available":96076517376,"internally_reserved":0,"allocated":223571968,"data_stored":223290140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790047,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572384,"kb_used_data":125568,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93799456,"statfs":{"total":96636764160,"available":96050642944,"internally_reserved":0,"allocated":128581632,"data_stored":128259597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633972,"kb_used_data":196956,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93737868,"statfs":{"total":96636764160,"available":95987576832,"internally_reserved":0,"allocated":201682944,"data_stored":201401111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200097792,"data_stored":200061662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127602688,"data_stored":127526356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222167040,"data_stored":222126838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:13.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:13.597 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:13.597 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:13.623 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-11T01:23:12.619468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":174514953,"num_objects":42618,"num_object_clones":0,"num_object_copies":85236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42618,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":242627,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11985,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1753472,"kb_used_data":540856,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281362048,"statfs":{"total":289910292480,"available":288114737152,"internally_reserved":0,"allocated":553836544,"data_stored":552950848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1241631751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,4,8,13,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23093248,"num_objects":-5638,"num_object_clones":0,"num_object_copies":-11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004060"},"pg_stats":[{"pgid":"2.f","version":"43'17534","reported_seq":17758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.332928+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:08.332928+0000","last_peered":"2024-09-11T01:23:08.332928+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:08.332928+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11075584,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17534,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17853","reported_seq":17933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.067946+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:11.067946+0000","last_peered":"2024-09-11T01:23:11.067946+0000","last_clean":"2024-09-11T01:23:11.067946+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:11.067946+0000","last_undegraded":"2024-09-11T01:23:11.067946+0000","last_fullsized":"2024-09-11T01:23:11.067946+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10933190,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15951,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17503","reported_seq":17672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.001750+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:11.001750+0000","last_peered":"2024-09-11T01:23:11.001750+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:11.001750+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10412056,"num_objects":2543,"num_object_clones":0,"num_object_copies":5086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15564,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17708","reported_seq":17754,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:10.996775+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:10.996775+0000","last_peered":"2024-09-11T01:23:10.996775+0000","last_clean":"2024-09-11T01:23:10.996775+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:10.996775+0000","last_undegraded":"2024-09-11T01:23:10.996775+0000","last_fullsized":"2024-09-11T01:23:10.996775+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10870784,"num_objects":2654,"num_object_clones":0,"num_object_copies":5308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17708,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17548","reported_seq":17671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:08.337812+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:08.337812+0000","last_peered":"2024-09-11T01:23:08.337812+0000","last_clean":"2024-09-11T01:23:08.337812+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:08.337812+0000","last_undegraded":"2024-09-11T01:23:08.337812+0000","last_fullsized":"2024-09-11T01:23:08.337812+0000","mapping_epoch":31,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11178467,"num_objects":2730,"num_object_clones":0,"num_object_copies":5460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14346,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17575","reported_seq":17839,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.428582+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:08.428582+0000","last_peered":"2024-09-11T01:23:08.428582+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:08.428582+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10781155,"num_objects":2633,"num_object_clones":0,"num_object_copies":5266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14063,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1416},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18049","reported_seq":18338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.376615+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:08.376615+0000","last_peered":"2024-09-11T01:23:08.376615+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:08.376615+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11318214,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14802,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1441},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17704","reported_seq":17825,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:08.309315+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:08.309315+0000","last_peered":"2024-09-11T01:23:08.309315+0000","last_clean":"2024-09-11T01:23:08.309315+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:08.309315+0000","last_undegraded":"2024-09-11T01:23:08.309315+0000","last_fullsized":"2024-09-11T01:23:08.309315+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11141120,"num_objects":2720,"num_object_clones":0,"num_object_copies":5440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14657,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17694","reported_seq":26038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.552912+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:11.552912+0000","last_peered":"2024-09-11T01:23:11.552912+0000","last_clean":"2024-09-11T01:23:11.552912+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:11.552912+0000","last_undegraded":"2024-09-11T01:23:11.552912+0000","last_fullsized":"2024-09-11T01:23:11.552912+0000","mapping_epoch":40,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10465763,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15561,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17847","reported_seq":18125,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.393581+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:08.393581+0000","last_peered":"2024-09-11T01:23:08.393581+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:08.393581+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11251712,"num_objects":2747,"num_object_clones":0,"num_object_copies":5494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14211,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1283}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17498","reported_seq":23830,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.101453+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:11.101453+0000","last_peered":"2024-09-11T01:23:11.101453+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:11.101453+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818019,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15267,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3118,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1732}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17974","reported_seq":18129,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.074755+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:11.074755+0000","last_peered":"2024-09-11T01:23:11.074755+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:11.074755+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10600448,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15611,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1559}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17755","reported_seq":26227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.538997+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:11.538997+0000","last_peered":"2024-09-11T01:23:11.538997+0000","last_clean":"2024-09-11T01:23:11.538997+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:11.538997+0000","last_undegraded":"2024-09-11T01:23:11.538997+0000","last_fullsized":"2024-09-11T01:23:11.538997+0000","mapping_epoch":42,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10637312,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'17980","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.459771+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:08.459771+0000","last_peered":"2024-09-11T01:23:08.459771+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:08.459771+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10919936,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14373,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17860","reported_seq":18161,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.420189+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:08.420189+0000","last_peered":"2024-09-11T01:23:08.420189+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:08.420189+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11223040,"num_objects":2740,"num_object_clones":0,"num_object_copies":5480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14395,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1325}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17480","reported_seq":17720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.362684+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:08.362684+0000","last_peered":"2024-09-11T01:23:08.362684+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:08.362684+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10888134,"num_objects":2660,"num_object_clones":0,"num_object_copies":5320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2660,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14137,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1442},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":174514934,"num_objects":42616,"num_object_clones":0,"num_object_copies":85232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42616,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":242625,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11984,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":549867520,"data_stored":549714856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547116,"kb_used_data":218332,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93824724,"statfs":{"total":96636764160,"available":96076517376,"internally_reserved":0,"allocated":223571968,"data_stored":223290140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790047,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572384,"kb_used_data":125568,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93799456,"statfs":{"total":96636764160,"available":96050642944,"internally_reserved":0,"allocated":128581632,"data_stored":128259597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":633972,"kb_used_data":196956,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93737868,"statfs":{"total":96636764160,"available":95987576832,"internally_reserved":0,"allocated":201682944,"data_stored":201401111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200097792,"data_stored":200061662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127602688,"data_stored":127526356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222167040,"data_stored":222126838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17534", 'reported_seq': 17758, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.332928+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:08.332928+0000', 'last_peered': '2024-09-11T01:23:08.332928+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:08.332928+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11075584, 'num_objects': 2704, 'num_object_clones': 0, 'num_object_copies': 5408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17534, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1344}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17503", 'reported_seq': 17672, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:11.001750+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:11.001750+0000', 'last_peered': '2024-09-11T01:23:11.001750+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:11.001750+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10412056, 'num_objects': 2543, 'num_object_clones': 0, 'num_object_copies': 5086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2543, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15564, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1420}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17575", 'reported_seq': 17839, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.428582+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:08.428582+0000', 'last_peered': '2024-09-11T01:23:08.428582+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:08.428582+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7500", 'ondisk_log_start': "39'7500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10781155, 'num_objects': 2633, 'num_object_clones': 0, 'num_object_copies': 5266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2633, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14063, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1416}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:13.625 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'18049", 'reported_seq': 18338, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.376615+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:08.376615+0000', 'last_peered': '2024-09-11T01:23:08.376615+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:08.376615+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'8000", 'ondisk_log_start': "39'8000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11318214, 'num_objects': 2765, 'num_object_clones': 0, 'num_object_copies': 5530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2765, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 14802, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1441}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'17847", 'reported_seq': 18125, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.393581+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:08.393581+0000', 'last_peered': '2024-09-11T01:23:08.393581+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:08.393581+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11251712, 'num_objects': 2747, 'num_object_clones': 0, 'num_object_copies': 5494, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2747, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14211, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1283}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17498", 'reported_seq': 23830, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:11.101453+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:11.101453+0000', 'last_peered': '2024-09-11T01:23:11.101453+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:11.101453+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7400", 'ondisk_log_start': "39'7400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10818019, 'num_objects': 2642, 'num_object_clones': 0, 'num_object_copies': 5284, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 910, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2642, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15267, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3118, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1732}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'17974", 'reported_seq': 18129, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:11.074755+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:11.074755+0000', 'last_peered': '2024-09-11T01:23:11.074755+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:11.074755+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10600448, 'num_objects': 2588, 'num_object_clones': 0, 'num_object_copies': 5176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15611, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1559}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'17980", 'reported_seq': 18266, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.459771+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:08.459771+0000', 'last_peered': '2024-09-11T01:23:08.459771+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:08.459771+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10919936, 'num_objects': 2666, 'num_object_clones': 0, 'num_object_copies': 5332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14373, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:13.626 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'17860", 'reported_seq': 18161, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.420189+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:08.420189+0000', 'last_peered': '2024-09-11T01:23:08.420189+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:08.420189+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11223040, 'num_objects': 2740, 'num_object_clones': 0, 'num_object_copies': 5480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14395, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1325}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.627 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:13.627 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17480", 'reported_seq': 17720, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.362684+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:08.362684+0000', 'last_peered': '2024-09-11T01:23:08.362684+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:08.362684+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7400", 'ondisk_log_start': "39'7400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10888134, 'num_objects': 2660, 'num_object_clones': 0, 'num_object_copies': 5320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2660, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14137, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1442}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:13.627 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:14.339 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:14.339 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37650/85236 objects degraded (44.171%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":594,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":42618,"data_bytes":174514953,"bytes_used":1795555328,"bytes_avail":288114737152,"bytes_total":289910292480,"degraded_objects":37650,"degraded_total":85236,"degraded_ratio":0.44171476840771506,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":469},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:14.366 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:14.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:14.942 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:14.942 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:14.967 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-11T01:23:14.620178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":169202441,"num_objects":41321,"num_object_clones":0,"num_object_copies":82642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41321,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":243924,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11985,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161062,"ondisk_log_size":161062,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1734128,"kb_used_data":533992,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281381392,"statfs":{"total":289910292480,"available":288134545408,"internally_reserved":0,"allocated":546807808,"data_stored":545922112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1228852231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,6,7,11,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24236032,"num_objects":-5917,"num_object_clones":0,"num_object_copies":-11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5917,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004332"},"pg_stats":[{"pgid":"2.f","version":"43'17682","reported_seq":17905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.484757+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:13.484757+0000","last_peered":"2024-09-11T01:23:13.484757+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:13.484757+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10469376,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17682,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1492}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17853","reported_seq":17933,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.067946+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:11.067946+0000","last_peered":"2024-09-11T01:23:11.067946+0000","last_clean":"2024-09-11T01:23:11.067946+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:11.067946+0000","last_undegraded":"2024-09-11T01:23:11.067946+0000","last_fullsized":"2024-09-11T01:23:11.067946+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10933190,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15951,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17503","reported_seq":17672,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.001750+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:11.001750+0000","last_peered":"2024-09-11T01:23:11.001750+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:11.001750+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7500","ondisk_log_start":"39'7500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10412056,"num_objects":2543,"num_object_clones":0,"num_object_copies":5086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15564,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17708","reported_seq":17754,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:10.996775+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:10.996775+0000","last_peered":"2024-09-11T01:23:10.996775+0000","last_clean":"2024-09-11T01:23:10.996775+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:10.996775+0000","last_undegraded":"2024-09-11T01:23:10.996775+0000","last_fullsized":"2024-09-11T01:23:10.996775+0000","mapping_epoch":31,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10870784,"num_objects":2654,"num_object_clones":0,"num_object_copies":5308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17708,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17685","reported_seq":17808,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:13.396286+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:13.396286+0000","last_peered":"2024-09-11T01:23:13.396286+0000","last_clean":"2024-09-11T01:23:13.396286+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:13.396286+0000","last_undegraded":"2024-09-11T01:23:13.396286+0000","last_fullsized":"2024-09-11T01:23:13.396286+0000","mapping_epoch":31,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10617315,"num_objects":2593,"num_object_clones":0,"num_object_copies":5186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14483,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17700","reported_seq":17967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.410820+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:13.410820+0000","last_peered":"2024-09-11T01:23:13.410820+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:13.410820+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269155,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14188,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18198","reported_seq":18487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.438674+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:13.438674+0000","last_peered":"2024-09-11T01:23:13.438674+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:13.438674+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10707910,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14951,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1590},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17864","reported_seq":17984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:13.460836+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:13.460836+0000","last_peered":"2024-09-11T01:23:13.460836+0000","last_clean":"2024-09-11T01:23:13.460836+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:13.460836+0000","last_undegraded":"2024-09-11T01:23:13.460836+0000","last_fullsized":"2024-09-11T01:23:13.460836+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2560,"num_object_clones":0,"num_object_copies":5120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14817,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17694","reported_seq":26038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.552912+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:11.552912+0000","last_peered":"2024-09-11T01:23:11.552912+0000","last_clean":"2024-09-11T01:23:11.552912+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:11.552912+0000","last_undegraded":"2024-09-11T01:23:11.552912+0000","last_fullsized":"2024-09-11T01:23:11.552912+0000","mapping_epoch":40,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10465763,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15561,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17996","reported_seq":18276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.340134+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:13.340134+0000","last_peered":"2024-09-11T01:23:13.340134+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:13.340134+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10641408,"num_objects":2598,"num_object_clones":0,"num_object_copies":5196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14360,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17498","reported_seq":23830,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.101453+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:11.101453+0000","last_peered":"2024-09-11T01:23:11.101453+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:11.101453+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7400","ondisk_log_start":"39'7400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10818019,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15267,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3118,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1732}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'17974","reported_seq":18129,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:11.074755+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:11.074755+0000","last_peered":"2024-09-11T01:23:11.074755+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:11.074755+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10600448,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15611,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1559}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17755","reported_seq":26227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.538997+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:11.538997+0000","last_peered":"2024-09-11T01:23:11.538997+0000","last_clean":"2024-09-11T01:23:11.538997+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:11.538997+0000","last_undegraded":"2024-09-11T01:23:11.538997+0000","last_fullsized":"2024-09-11T01:23:11.538997+0000","mapping_epoch":42,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10637312,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18137","reported_seq":18423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.461877+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:13.461877+0000","last_peered":"2024-09-11T01:23:13.461877+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:13.461877+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10276864,"num_objects":2509,"num_object_clones":0,"num_object_copies":5018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14530,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17999","reported_seq":18298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.468279+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:13.468279+0000","last_peered":"2024-09-11T01:23:13.468279+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:13.468279+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10653696,"num_objects":2601,"num_object_clones":0,"num_object_copies":5202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17613","reported_seq":17853,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.400621+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:13.400621+0000","last_peered":"2024-09-11T01:23:13.400621+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:13.400621+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10343366,"num_objects":2527,"num_object_clones":0,"num_object_copies":5054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14270,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1575},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":169202422,"num_objects":41319,"num_object_clones":0,"num_object_copies":82638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41319,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":243922,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11984,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":542867456,"data_stored":542714792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161059,"ondisk_log_size":161059,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547116,"kb_used_data":218332,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93824724,"statfs":{"total":96636764160,"available":96076517376,"internally_reserved":0,"allocated":223571968,"data_stored":223290140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790047,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572384,"kb_used_data":125568,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93799456,"statfs":{"total":96636764160,"available":96050642944,"internally_reserved":0,"allocated":128581632,"data_stored":128259597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614628,"kb_used_data":190092,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93757212,"statfs":{"total":96636764160,"available":96007385088,"internally_reserved":0,"allocated":194654208,"data_stored":194372375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193097728,"data_stored":193061598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127602688,"data_stored":127526356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":222167040,"data_stored":222126838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:17.969 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:18.575 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:18.576 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:18.599 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-11T01:23:16.620657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166486793,"num_objects":40658,"num_object_clones":0,"num_object_copies":81316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40658,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":244587,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12134,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161025,"ondisk_log_size":161025,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1716260,"kb_used_data":528604,"kb_used_omap":12,"kb_used_meta":1187571,"kb_avail":281399260,"statfs":{"total":289910292480,"available":288152842240,"internally_reserved":0,"allocated":541290496,"data_stored":540404800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1216072711},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,4,12,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21606400,"num_objects":-5275,"num_object_clones":0,"num_object_copies":-10550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5275,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":379,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004344"},"pg_stats":[{"pgid":"2.f","version":"43'17682","reported_seq":17905,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.484757+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:13.484757+0000","last_peered":"2024-09-11T01:23:13.484757+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:13.484757+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10469376,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17682,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1492}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17990","reported_seq":18071,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.011299+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:16.011299+0000","last_peered":"2024-09-11T01:23:16.011299+0000","last_clean":"2024-09-11T01:23:16.011299+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:16.011299+0000","last_undegraded":"2024-09-11T01:23:16.011299+0000","last_fullsized":"2024-09-11T01:23:16.011299+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10372038,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16088,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17640","reported_seq":17809,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.114076+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:16.114076+0000","last_peered":"2024-09-11T01:23:16.114076+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:16.114076+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9850904,"num_objects":2406,"num_object_clones":0,"num_object_copies":4812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15701,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17840","reported_seq":17886,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.107895+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:16.107895+0000","last_peered":"2024-09-11T01:23:16.107895+0000","last_clean":"2024-09-11T01:23:16.107895+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:16.107895+0000","last_undegraded":"2024-09-11T01:23:16.107895+0000","last_fullsized":"2024-09-11T01:23:16.107895+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17840,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17685","reported_seq":17808,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:13.396286+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:13.396286+0000","last_peered":"2024-09-11T01:23:13.396286+0000","last_clean":"2024-09-11T01:23:13.396286+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:13.396286+0000","last_undegraded":"2024-09-11T01:23:13.396286+0000","last_fullsized":"2024-09-11T01:23:13.396286+0000","mapping_epoch":31,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10617315,"num_objects":2593,"num_object_clones":0,"num_object_copies":5186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14483,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17700","reported_seq":17967,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.410820+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:13.410820+0000","last_peered":"2024-09-11T01:23:13.410820+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:13.410820+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269155,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14188,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1541},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18198","reported_seq":18487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.438674+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:13.438674+0000","last_peered":"2024-09-11T01:23:13.438674+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:13.438674+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10707910,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14951,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1590},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'17864","reported_seq":17984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:13.460836+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:13.460836+0000","last_peered":"2024-09-11T01:23:13.460836+0000","last_clean":"2024-09-11T01:23:13.460836+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:13.460836+0000","last_undegraded":"2024-09-11T01:23:13.460836+0000","last_fullsized":"2024-09-11T01:23:13.460836+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10485760,"num_objects":2560,"num_object_clones":0,"num_object_copies":5120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14817,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17694","reported_seq":26038,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.552912+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:11.552912+0000","last_peered":"2024-09-11T01:23:11.552912+0000","last_clean":"2024-09-11T01:23:11.552912+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:11.552912+0000","last_undegraded":"2024-09-11T01:23:11.552912+0000","last_fullsized":"2024-09-11T01:23:11.552912+0000","mapping_epoch":40,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10465763,"num_objects":2556,"num_object_clones":0,"num_object_copies":5112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15561,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'17996","reported_seq":18276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.340134+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:13.340134+0000","last_peered":"2024-09-11T01:23:13.340134+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:13.340134+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10641408,"num_objects":2598,"num_object_clones":0,"num_object_copies":5196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14360,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17612","reported_seq":24242,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.113416+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:16.113416+0000","last_peered":"2024-09-11T01:23:16.113416+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:16.113416+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10351075,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15381,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3267,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1767}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18117","reported_seq":18272,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.026685+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:16.026685+0000","last_peered":"2024-09-11T01:23:16.026685+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:16.026685+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10014720,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17755","reported_seq":26227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:11.538997+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:11.538997+0000","last_peered":"2024-09-11T01:23:11.538997+0000","last_clean":"2024-09-11T01:23:11.538997+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:11.538997+0000","last_undegraded":"2024-09-11T01:23:11.538997+0000","last_fullsized":"2024-09-11T01:23:11.538997+0000","mapping_epoch":42,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10637312,"num_objects":2597,"num_object_clones":0,"num_object_copies":5194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14445,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18137","reported_seq":18423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.461877+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:13.461877+0000","last_peered":"2024-09-11T01:23:13.461877+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:13.461877+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10276864,"num_objects":2509,"num_object_clones":0,"num_object_copies":5018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2509,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14530,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'17999","reported_seq":18298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.468279+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:13.468279+0000","last_peered":"2024-09-11T01:23:13.468279+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:13.468279+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10653696,"num_objects":2601,"num_object_clones":0,"num_object_copies":5202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14534,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17613","reported_seq":17853,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.400621+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:13.400621+0000","last_peered":"2024-09-11T01:23:13.400621+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:13.400621+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10343366,"num_objects":2527,"num_object_clones":0,"num_object_copies":5054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14270,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1575},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166486774,"num_objects":40656,"num_object_clones":0,"num_object_copies":81312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40656,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":244585,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12133,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":538349568,"data_stored":538196904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161022,"ondisk_log_size":161022,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529248,"kb_used_data":212944,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93842592,"statfs":{"total":96636764160,"available":96094814208,"internally_reserved":0,"allocated":218054656,"data_stored":217772828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790047,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572384,"kb_used_data":125568,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93799456,"statfs":{"total":96636764160,"available":96050642944,"internally_reserved":0,"allocated":128581632,"data_stored":128259597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614628,"kb_used_data":190092,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93757212,"statfs":{"total":96636764160,"available":96007385088,"internally_reserved":0,"allocated":194654208,"data_stored":194372375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":193097728,"data_stored":193061598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":127602688,"data_stored":127526356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217649152,"data_stored":217608950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:18.601 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:19.184 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:19.187 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:19.206 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-11T01:23:18.621278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":159810313,"num_objects":39028,"num_object_clones":0,"num_object_copies":78056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39028,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":246217,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12134,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1710708,"kb_used_data":514736,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281404812,"statfs":{"total":289910292480,"available":288158527488,"internally_reserved":0,"allocated":527089664,"data_stored":526203968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1224592391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,5,8,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24190976,"num_objects":-5906,"num_object_clones":0,"num_object_copies":-11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004083"},"pg_stats":[{"pgid":"2.f","version":"43'17829","reported_seq":18053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.435751+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:18.435751+0000","last_peered":"2024-09-11T01:23:18.435751+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:18.435751+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17829,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1639}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17990","reported_seq":18071,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.011299+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:16.011299+0000","last_peered":"2024-09-11T01:23:16.011299+0000","last_clean":"2024-09-11T01:23:16.011299+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:16.011299+0000","last_undegraded":"2024-09-11T01:23:16.011299+0000","last_fullsized":"2024-09-11T01:23:16.011299+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10372038,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16088,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17640","reported_seq":17809,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.114076+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:16.114076+0000","last_peered":"2024-09-11T01:23:16.114076+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:16.114076+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9850904,"num_objects":2406,"num_object_clones":0,"num_object_copies":4812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15701,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17840","reported_seq":17886,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.107895+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:16.107895+0000","last_peered":"2024-09-11T01:23:16.107895+0000","last_clean":"2024-09-11T01:23:16.107895+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:16.107895+0000","last_undegraded":"2024-09-11T01:23:16.107895+0000","last_fullsized":"2024-09-11T01:23:16.107895+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17840,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17845","reported_seq":17968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:18.494908+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:18.494908+0000","last_peered":"2024-09-11T01:23:18.494908+0000","last_clean":"2024-09-11T01:23:18.494908+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:18.494908+0000","last_undegraded":"2024-09-11T01:23:18.494908+0000","last_fullsized":"2024-09-11T01:23:18.494908+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9961955,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14643,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17849","reported_seq":18117,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.461836+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:18.461836+0000","last_peered":"2024-09-11T01:23:18.461836+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:18.461836+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9658851,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14337,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1690},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18358","reported_seq":18647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.472253+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:18.472253+0000","last_peered":"2024-09-11T01:23:18.472253+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:18.472253+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10052550,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15111,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1750},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18000","reported_seq":18121,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:18.502438+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:18.502438+0000","last_peered":"2024-09-11T01:23:18.502438+0000","last_clean":"2024-09-11T01:23:18.502438+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:18.502438+0000","last_undegraded":"2024-09-11T01:23:18.502438+0000","last_fullsized":"2024-09-11T01:23:18.502438+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9928704,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17842","reported_seq":26186,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.511633+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:16.511633+0000","last_peered":"2024-09-11T01:23:16.511633+0000","last_clean":"2024-09-11T01:23:16.511633+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:16.511633+0000","last_undegraded":"2024-09-11T01:23:16.511633+0000","last_fullsized":"2024-09-11T01:23:16.511633+0000","mapping_epoch":40,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9859555,"num_objects":2408,"num_object_clones":0,"num_object_copies":4816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15709,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18153","reported_seq":18433,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.458331+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:18.458331+0000","last_peered":"2024-09-11T01:23:18.458331+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:18.458331+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9998336,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14517,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17612","reported_seq":24242,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.113416+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:16.113416+0000","last_peered":"2024-09-11T01:23:16.113416+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:16.113416+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10351075,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15381,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3267,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1767}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18117","reported_seq":18272,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.026685+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:16.026685+0000","last_peered":"2024-09-11T01:23:16.026685+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:16.026685+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10014720,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17888","reported_seq":26360,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.564996+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:16.564996+0000","last_peered":"2024-09-11T01:23:16.564996+0000","last_clean":"2024-09-11T01:23:16.564996+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:16.564996+0000","last_undegraded":"2024-09-11T01:23:16.564996+0000","last_fullsized":"2024-09-11T01:23:16.564996+0000","mapping_epoch":42,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14578,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18294","reported_seq":18580,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.474197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:18.474197+0000","last_peered":"2024-09-11T01:23:18.474197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:18.474197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9633792,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14687,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18128","reported_seq":18427,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.536865+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:18.536865+0000","last_peered":"2024-09-11T01:23:18.536865+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:18.536865+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10125312,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14663,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17767","reported_seq":18007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.405578+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:18.405578+0000","last_peered":"2024-09-11T01:23:18.405578+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:18.405578+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14424,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1729},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":159810294,"num_objects":39026,"num_object_clones":0,"num_object_copies":78052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39026,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":246215,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12133,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524226560,"data_stored":524073896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529248,"kb_used_data":212944,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93842592,"statfs":{"total":96636764160,"available":96094814208,"internally_reserved":0,"allocated":218054656,"data_stored":217772828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790048,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569772,"kb_used_data":118812,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93802068,"statfs":{"total":96636764160,"available":96053317632,"internally_reserved":0,"allocated":121663488,"data_stored":121341453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611688,"kb_used_data":182980,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93760152,"statfs":{"total":96636764160,"available":96010395648,"internally_reserved":0,"allocated":187371520,"data_stored":187089687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185835520,"data_stored":185799390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":120741888,"data_stored":120665556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217649152,"data_stored":217608950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:19.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:19.207 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17829", 'reported_seq': 18053, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.435751+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:18.435751+0000', 'last_peered': '2024-09-11T01:23:18.435751+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:18.435751+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9867264, 'num_objects': 2409, 'num_object_clones': 0, 'num_object_copies': 4818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17829, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1639}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.207 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17640", 'reported_seq': 17809, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:16.114076+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:16.114076+0000', 'last_peered': '2024-09-11T01:23:16.114076+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:16.114076+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7600", 'ondisk_log_start': "39'7600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9850904, 'num_objects': 2406, 'num_object_clones': 0, 'num_object_copies': 4812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2406, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15701, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17849", 'reported_seq': 18117, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.461836+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:18.461836+0000', 'last_peered': '2024-09-11T01:23:18.461836+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:18.461836+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9658851, 'num_objects': 2359, 'num_object_clones': 0, 'num_object_copies': 4718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2359, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14337, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1690}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'18358", 'reported_seq': 18647, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.472253+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:18.472253+0000', 'last_peered': '2024-09-11T01:23:18.472253+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:18.472253+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'8300", 'ondisk_log_start': "39'8300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10052550, 'num_objects': 2456, 'num_object_clones': 0, 'num_object_copies': 4912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2456, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15111, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1750}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'18153", 'reported_seq': 18433, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.458331+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:18.458331+0000', 'last_peered': '2024-09-11T01:23:18.458331+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:18.458331+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'8100", 'ondisk_log_start': "39'8100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9998336, 'num_objects': 2441, 'num_object_clones': 0, 'num_object_copies': 4882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2441, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14517, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1589}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17612", 'reported_seq': 24242, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:16.113416+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:16.113416+0000', 'last_peered': '2024-09-11T01:23:16.113416+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:16.113416+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7600", 'ondisk_log_start': "39'7600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10351075, 'num_objects': 2528, 'num_object_clones': 0, 'num_object_copies': 5056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 761, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2528, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15381, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3267, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1767}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:19.208 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'18117", 'reported_seq': 18272, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:16.026685+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:16.026685+0000', 'last_peered': '2024-09-11T01:23:16.026685+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:16.026685+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'8100", 'ondisk_log_start': "39'8100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10014720, 'num_objects': 2445, 'num_object_clones': 0, 'num_object_copies': 4890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2445, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15754, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1702}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'18294", 'reported_seq': 18580, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.474197+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:18.474197+0000', 'last_peered': '2024-09-11T01:23:18.474197+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:18.474197+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9633792, 'num_objects': 2352, 'num_object_clones': 0, 'num_object_copies': 4704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14687, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1765}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'18128", 'reported_seq': 18427, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.536865+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:18.536865+0000', 'last_peered': '2024-09-11T01:23:18.536865+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:18.536865+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'8100", 'ondisk_log_start': "39'8100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10125312, 'num_objects': 2472, 'num_object_clones': 0, 'num_object_copies': 4944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14663, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:19.209 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17767", 'reported_seq': 18007, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.405578+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:18.405578+0000', 'last_peered': '2024-09-11T01:23:18.405578+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:18.405578+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7700", 'ondisk_log_start': "39'7700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9712582, 'num_objects': 2373, 'num_object_clones': 0, 'num_object_copies': 4746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2373, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14424, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1729}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:19.930 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:19.930 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37501/78056 objects degraded (48.044%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":600,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":39028,"data_bytes":159810313,"bytes_used":1751764992,"bytes_avail":288158527488,"bytes_total":289910292480,"degraded_objects":37501,"degraded_total":78056,"degraded_ratio":0.48043712206620887,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":491},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:19.954 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:19.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:20.544 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:20.546 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:20.570 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-11T01:23:18.621278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":159810313,"num_objects":39028,"num_object_clones":0,"num_object_copies":78056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39028,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":246217,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12134,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1710708,"kb_used_data":514736,"kb_used_omap":12,"kb_used_meta":1195891,"kb_avail":281404812,"statfs":{"total":289910292480,"available":288158527488,"internally_reserved":0,"allocated":527089664,"data_stored":526203968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1224592391},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,6,5,8,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24190976,"num_objects":-5906,"num_object_clones":0,"num_object_copies":-11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-260,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004083"},"pg_stats":[{"pgid":"2.f","version":"43'17829","reported_seq":18053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.435751+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:18.435751+0000","last_peered":"2024-09-11T01:23:18.435751+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:18.435751+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17829,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1639}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'17990","reported_seq":18071,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.011299+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:16.011299+0000","last_peered":"2024-09-11T01:23:16.011299+0000","last_clean":"2024-09-11T01:23:16.011299+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:16.011299+0000","last_undegraded":"2024-09-11T01:23:16.011299+0000","last_fullsized":"2024-09-11T01:23:16.011299+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10372038,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16088,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17640","reported_seq":17809,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.114076+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:16.114076+0000","last_peered":"2024-09-11T01:23:16.114076+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:16.114076+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9850904,"num_objects":2406,"num_object_clones":0,"num_object_copies":4812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15701,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17840","reported_seq":17886,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.107895+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:16.107895+0000","last_peered":"2024-09-11T01:23:16.107895+0000","last_clean":"2024-09-11T01:23:16.107895+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:16.107895+0000","last_undegraded":"2024-09-11T01:23:16.107895+0000","last_fullsized":"2024-09-11T01:23:16.107895+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10330112,"num_objects":2522,"num_object_clones":0,"num_object_copies":5044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17840,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17845","reported_seq":17968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:18.494908+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:18.494908+0000","last_peered":"2024-09-11T01:23:18.494908+0000","last_clean":"2024-09-11T01:23:18.494908+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:18.494908+0000","last_undegraded":"2024-09-11T01:23:18.494908+0000","last_fullsized":"2024-09-11T01:23:18.494908+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9961955,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14643,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17849","reported_seq":18117,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.461836+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:18.461836+0000","last_peered":"2024-09-11T01:23:18.461836+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:18.461836+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9658851,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14337,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1690},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18358","reported_seq":18647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.472253+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:18.472253+0000","last_peered":"2024-09-11T01:23:18.472253+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:18.472253+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10052550,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15111,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1750},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18000","reported_seq":18121,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:18.502438+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:18.502438+0000","last_peered":"2024-09-11T01:23:18.502438+0000","last_clean":"2024-09-11T01:23:18.502438+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:18.502438+0000","last_undegraded":"2024-09-11T01:23:18.502438+0000","last_fullsized":"2024-09-11T01:23:18.502438+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9928704,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680005+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:20:45.680005+0000","last_peered":"2024-09-11T01:20:45.680005+0000","last_clean":"2024-09-11T01:20:45.680005+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:20:45.680005+0000","last_undegraded":"2024-09-11T01:20:45.680005+0000","last_fullsized":"2024-09-11T01:20:45.680005+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'17842","reported_seq":26186,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.511633+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:16.511633+0000","last_peered":"2024-09-11T01:23:16.511633+0000","last_clean":"2024-09-11T01:23:16.511633+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:16.511633+0000","last_undegraded":"2024-09-11T01:23:16.511633+0000","last_fullsized":"2024-09-11T01:23:16.511633+0000","mapping_epoch":40,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9859555,"num_objects":2408,"num_object_clones":0,"num_object_copies":4816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2408,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15709,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18153","reported_seq":18433,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.458331+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:18.458331+0000","last_peered":"2024-09-11T01:23:18.458331+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:18.458331+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9998336,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14517,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17612","reported_seq":24242,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.113416+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:16.113416+0000","last_peered":"2024-09-11T01:23:16.113416+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:16.113416+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7600","ondisk_log_start":"39'7600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10351075,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15381,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3267,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1767}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18117","reported_seq":18272,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:16.026685+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:16.026685+0000","last_peered":"2024-09-11T01:23:16.026685+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:16.026685+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10014720,"num_objects":2445,"num_object_clones":0,"num_object_copies":4890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15754,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1702}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'17888","reported_seq":26360,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:16.564996+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:16.564996+0000","last_peered":"2024-09-11T01:23:16.564996+0000","last_clean":"2024-09-11T01:23:16.564996+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:16.564996+0000","last_undegraded":"2024-09-11T01:23:16.564996+0000","last_fullsized":"2024-09-11T01:23:16.564996+0000","mapping_epoch":42,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10092544,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14578,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18294","reported_seq":18580,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.474197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:18.474197+0000","last_peered":"2024-09-11T01:23:18.474197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:18.474197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9633792,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14687,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18128","reported_seq":18427,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.536865+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:18.536865+0000","last_peered":"2024-09-11T01:23:18.536865+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:18.536865+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10125312,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14663,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17767","reported_seq":18007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.405578+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:18.405578+0000","last_peered":"2024-09-11T01:23:18.405578+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:18.405578+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14424,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1729},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":159810294,"num_objects":39026,"num_object_clones":0,"num_object_copies":78052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37501,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39026,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":246215,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12133,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524226560,"data_stored":524073896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529248,"kb_used_data":212944,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93842592,"statfs":{"total":96636764160,"available":96094814208,"internally_reserved":0,"allocated":218054656,"data_stored":217772828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790048,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569772,"kb_used_data":118812,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93802068,"statfs":{"total":96636764160,"available":96053317632,"internally_reserved":0,"allocated":121663488,"data_stored":121341453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611688,"kb_used_data":182980,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93760152,"statfs":{"total":96636764160,"available":96010395648,"internally_reserved":0,"allocated":187371520,"data_stored":187089687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185835520,"data_stored":185799390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":120741888,"data_stored":120665556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":217649152,"data_stored":217608950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:23.573 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:24.161 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:24.164 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:24.183 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-11T01:23:22.622531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":155697929,"num_objects":38024,"num_object_clones":0,"num_object_copies":76048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38024,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":247221,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12247,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1695024,"kb_used_data":503208,"kb_used_omap":12,"kb_used_meta":1191731,"kb_avail":281420496,"statfs":{"total":289910292480,"available":288174587904,"internally_reserved":0,"allocated":515284992,"data_stored":514399296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1220332551},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,7,5,11,17,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22921216,"num_objects":-5596,"num_object_clones":0,"num_object_copies":-11192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5596,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003707"},"pg_stats":[{"pgid":"2.f","version":"43'17829","reported_seq":18053,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.435751+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:18.435751+0000","last_peered":"2024-09-11T01:23:18.435751+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:18.435751+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9867264,"num_objects":2409,"num_object_clones":0,"num_object_copies":4818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17829,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1639}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18131","reported_seq":18212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.053388+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:21.053388+0000","last_peered":"2024-09-11T01:23:21.053388+0000","last_clean":"2024-09-11T01:23:21.053388+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:21.053388+0000","last_undegraded":"2024-09-11T01:23:21.053388+0000","last_fullsized":"2024-09-11T01:23:21.053388+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9794502,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16229,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17778","reported_seq":17946,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.102423+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:21.102423+0000","last_peered":"2024-09-11T01:23:21.102423+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:21.102423+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9285656,"num_objects":2268,"num_object_clones":0,"num_object_copies":4536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15839,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17995","reported_seq":18041,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.041834+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:21.041834+0000","last_peered":"2024-09-11T01:23:21.041834+0000","last_clean":"2024-09-11T01:23:21.041834+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:21.041834+0000","last_undegraded":"2024-09-11T01:23:21.041834+0000","last_fullsized":"2024-09-11T01:23:21.041834+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17995,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17845","reported_seq":17968,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:18.494908+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:18.494908+0000","last_peered":"2024-09-11T01:23:18.494908+0000","last_clean":"2024-09-11T01:23:18.494908+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:18.494908+0000","last_undegraded":"2024-09-11T01:23:18.494908+0000","last_fullsized":"2024-09-11T01:23:18.494908+0000","mapping_epoch":31,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9961955,"num_objects":2433,"num_object_clones":0,"num_object_copies":4866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2433,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14643,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17849","reported_seq":18117,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.461836+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:18.461836+0000","last_peered":"2024-09-11T01:23:18.461836+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:18.461836+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9658851,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14337,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1690},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18358","reported_seq":18647,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.472253+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:18.472253+0000","last_peered":"2024-09-11T01:23:18.472253+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:18.472253+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10052550,"num_objects":2456,"num_object_clones":0,"num_object_copies":4912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2456,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15111,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1750},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18000","reported_seq":18121,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:18.502438+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:18.502438+0000","last_peered":"2024-09-11T01:23:18.502438+0000","last_clean":"2024-09-11T01:23:18.502438+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:18.502438+0000","last_undegraded":"2024-09-11T01:23:18.502438+0000","last_fullsized":"2024-09-11T01:23:18.502438+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9928704,"num_objects":2424,"num_object_clones":0,"num_object_copies":4848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14953,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18000","reported_seq":26344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.700007+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:21.700007+0000","last_peered":"2024-09-11T01:23:21.700007+0000","last_clean":"2024-09-11T01:23:21.700007+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:21.700007+0000","last_undegraded":"2024-09-11T01:23:21.700007+0000","last_fullsized":"2024-09-11T01:23:21.700007+0000","mapping_epoch":40,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9212387,"num_objects":2250,"num_object_clones":0,"num_object_copies":4500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15867,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18153","reported_seq":18433,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.458331+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:18.458331+0000","last_peered":"2024-09-11T01:23:18.458331+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:18.458331+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9998336,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14517,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17749","reported_seq":24606,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.140180+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:21.140180+0000","last_peered":"2024-09-11T01:23:21.140180+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:21.140180+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9789923,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15518,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3380,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1743}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18252","reported_seq":18406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.086072+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:21.086072+0000","last_peered":"2024-09-11T01:23:21.086072+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:21.086072+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18028","reported_seq":26500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.581146+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:21.581146+0000","last_peered":"2024-09-11T01:23:21.581146+0000","last_clean":"2024-09-11T01:23:21.581146+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:21.581146+0000","last_undegraded":"2024-09-11T01:23:21.581146+0000","last_fullsized":"2024-09-11T01:23:21.581146+0000","mapping_epoch":42,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18294","reported_seq":18580,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.474197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:18.474197+0000","last_peered":"2024-09-11T01:23:18.474197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:18.474197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9633792,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14687,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18128","reported_seq":18427,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.536865+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:18.536865+0000","last_peered":"2024-09-11T01:23:18.536865+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:18.536865+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10125312,"num_objects":2472,"num_object_clones":0,"num_object_copies":4944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14663,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1593}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17767","reported_seq":18007,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.405578+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:18.405578+0000","last_peered":"2024-09-11T01:23:18.405578+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:18.405578+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14424,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1729},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":155697910,"num_objects":38022,"num_object_clones":0,"num_object_copies":76044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38022,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":247219,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12246,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512393216,"data_stored":512240552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528816,"kb_used_data":208352,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93843024,"statfs":{"total":96636764160,"available":96095256576,"internally_reserved":0,"allocated":213352448,"data_stored":213070620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790049,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554520,"kb_used_data":111876,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93817320,"statfs":{"total":96636764160,"available":96068935680,"internally_reserved":0,"allocated":114561024,"data_stored":114238989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611688,"kb_used_data":182980,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93760152,"statfs":{"total":96636764160,"available":96010395648,"internally_reserved":0,"allocated":187371520,"data_stored":187089687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185835520,"data_stored":185799390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113623040,"data_stored":113546708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212934656,"data_stored":212894454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:24.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:24.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:24.789 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:24.810 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-11T01:23:24.623433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150201097,"num_objects":36682,"num_object_clones":0,"num_object_copies":73364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36682,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":248563,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12247,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1696208,"kb_used_data":496088,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281419312,"statfs":{"total":289910292480,"available":288173375488,"internally_reserved":0,"allocated":507994112,"data_stored":507108416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1228852231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,5,6,5,13,20,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24313856,"num_objects":-5936,"num_object_clones":0,"num_object_copies":-11872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5936,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003965"},"pg_stats":[{"pgid":"2.f","version":"43'17985","reported_seq":18209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.537581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:23.537581+0000","last_peered":"2024-09-11T01:23:23.537581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:23.537581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1795}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18131","reported_seq":18212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.053388+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:21.053388+0000","last_peered":"2024-09-11T01:23:21.053388+0000","last_clean":"2024-09-11T01:23:21.053388+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:21.053388+0000","last_undegraded":"2024-09-11T01:23:21.053388+0000","last_fullsized":"2024-09-11T01:23:21.053388+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9794502,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16229,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17778","reported_seq":17946,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.102423+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:21.102423+0000","last_peered":"2024-09-11T01:23:21.102423+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:21.102423+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9285656,"num_objects":2268,"num_object_clones":0,"num_object_copies":4536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15839,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17995","reported_seq":18041,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.041834+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:21.041834+0000","last_peered":"2024-09-11T01:23:21.041834+0000","last_clean":"2024-09-11T01:23:21.041834+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:21.041834+0000","last_undegraded":"2024-09-11T01:23:21.041834+0000","last_fullsized":"2024-09-11T01:23:21.041834+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17995,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17992","reported_seq":18115,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.373606+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:23.373606+0000","last_peered":"2024-09-11T01:23:23.373606+0000","last_clean":"2024-09-11T01:23:23.373606+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:23.373606+0000","last_undegraded":"2024-09-11T01:23:23.373606+0000","last_fullsized":"2024-09-11T01:23:23.373606+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359843,"num_objects":2286,"num_object_clones":0,"num_object_copies":4572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14790,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17999","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.555384+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:23.555384+0000","last_peered":"2024-09-11T01:23:23.555384+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:23.555384+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9044451,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14487,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18509","reported_seq":18797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.542651+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:23.542651+0000","last_peered":"2024-09-11T01:23:23.542651+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:23.542651+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9434054,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15262,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1901},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18160","reported_seq":18281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.514831+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:23.514831+0000","last_peered":"2024-09-11T01:23:23.514831+0000","last_clean":"2024-09-11T01:23:23.514831+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:23.514831+0000","last_undegraded":"2024-09-11T01:23:23.514831+0000","last_fullsized":"2024-09-11T01:23:23.514831+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9273344,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15113,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18000","reported_seq":26344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.700007+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:21.700007+0000","last_peered":"2024-09-11T01:23:21.700007+0000","last_clean":"2024-09-11T01:23:21.700007+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:21.700007+0000","last_undegraded":"2024-09-11T01:23:21.700007+0000","last_fullsized":"2024-09-11T01:23:21.700007+0000","mapping_epoch":40,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9212387,"num_objects":2250,"num_object_clones":0,"num_object_copies":4500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15867,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18310","reported_seq":18586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.516755+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:23.516755+0000","last_peered":"2024-09-11T01:23:23.516755+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:23.516755+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17749","reported_seq":24606,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.140180+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:21.140180+0000","last_peered":"2024-09-11T01:23:21.140180+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:21.140180+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9789923,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15518,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3380,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1743}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18252","reported_seq":18406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.086072+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:21.086072+0000","last_peered":"2024-09-11T01:23:21.086072+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:21.086072+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18028","reported_seq":26500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.581146+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:21.581146+0000","last_peered":"2024-09-11T01:23:21.581146+0000","last_clean":"2024-09-11T01:23:21.581146+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:21.581146+0000","last_undegraded":"2024-09-11T01:23:21.581146+0000","last_fullsized":"2024-09-11T01:23:21.581146+0000","mapping_epoch":42,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18411","reported_seq":18697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.529197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:23.529197+0000","last_peered":"2024-09-11T01:23:23.529197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:23.529197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18276","reported_seq":18577,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.496017+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:23.496017+0000","last_peered":"2024-09-11T01:23:23.496017+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:23.496017+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14811,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1741}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17923","reported_seq":18162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.566406+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:23.566406+0000","last_peered":"2024-09-11T01:23:23.566406+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:23.566406+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073606,"num_objects":2217,"num_object_clones":0,"num_object_copies":4434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14580,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150201078,"num_objects":36680,"num_object_clones":0,"num_object_copies":73360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36680,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":248561,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12246,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":504950784,"data_stored":504798120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528816,"kb_used_data":208352,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93843024,"statfs":{"total":96636764160,"available":96095256576,"internally_reserved":0,"allocated":213352448,"data_stored":213070620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790049,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554520,"kb_used_data":111876,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93817320,"statfs":{"total":96636764160,"available":96068935680,"internally_reserved":0,"allocated":114561024,"data_stored":114238989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612872,"kb_used_data":175860,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93758968,"statfs":{"total":96636764160,"available":96009183232,"internally_reserved":0,"allocated":180080640,"data_stored":179798807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178393088,"data_stored":178356958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113623040,"data_stored":113546708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212934656,"data_stored":212894454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:24.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:24.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17985", 'reported_seq': 18209, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.537581+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:23.537581+0000', 'last_peered': '2024-09-11T01:23:23.537581+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:23.537581+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9228288, 'num_objects': 2253, 'num_object_clones': 0, 'num_object_copies': 4506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17985, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1795}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:24.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17778", 'reported_seq': 17946, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:21.102423+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:21.102423+0000', 'last_peered': '2024-09-11T01:23:21.102423+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:21.102423+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7700", 'ondisk_log_start': "39'7700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9285656, 'num_objects': 2268, 'num_object_clones': 0, 'num_object_copies': 4536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2268, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15839, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17999", 'reported_seq': 18266, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.555384+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:23.555384+0000', 'last_peered': '2024-09-11T01:23:23.555384+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:23.555384+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9044451, 'num_objects': 2209, 'num_object_clones': 0, 'num_object_copies': 4418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2209, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14487, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1840}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'18509", 'reported_seq': 18797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.542651+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:23.542651+0000', 'last_peered': '2024-09-11T01:23:23.542651+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:23.542651+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'8500", 'ondisk_log_start': "39'8500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9434054, 'num_objects': 2305, 'num_object_clones': 0, 'num_object_copies': 4610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2305, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15262, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1901}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'18310", 'reported_seq': 18586, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.516755+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:23.516755+0000', 'last_peered': '2024-09-11T01:23:23.516755+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:23.516755+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'8300", 'ondisk_log_start': "39'8300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9355264, 'num_objects': 2284, 'num_object_clones': 0, 'num_object_copies': 4568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14674, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1746}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17749", 'reported_seq': 24606, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:21.140180+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:21.140180+0000', 'last_peered': '2024-09-11T01:23:21.140180+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:21.140180+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7700", 'ondisk_log_start': "39'7700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9789923, 'num_objects': 2391, 'num_object_clones': 0, 'num_object_copies': 4782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 648, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2391, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15518, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3380, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1743}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:24.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'18252", 'reported_seq': 18406, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:21.086072+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:21.086072+0000', 'last_peered': '2024-09-11T01:23:21.086072+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:21.086072+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9461760, 'num_objects': 2310, 'num_object_clones': 0, 'num_object_copies': 4620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15889, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1837}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:24.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:24.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'18411", 'reported_seq': 18697, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.529197+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:23.529197+0000', 'last_peered': '2024-09-11T01:23:23.529197+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:23.529197+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'8400", 'ondisk_log_start': "39'8400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9154560, 'num_objects': 2235, 'num_object_clones': 0, 'num_object_copies': 4470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14804, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1882}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:24.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'18276", 'reported_seq': 18577, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.496017+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:23.496017+0000', 'last_peered': '2024-09-11T01:23:23.496017+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:23.496017+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9519104, 'num_objects': 2324, 'num_object_clones': 0, 'num_object_copies': 4648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14811, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:24.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17923", 'reported_seq': 18162, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.566406+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:23.566406+0000', 'last_peered': '2024-09-11T01:23:23.566406+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:23.566406+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9073606, 'num_objects': 2217, 'num_object_clones': 0, 'num_object_copies': 4434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2217, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14580, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1885}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:24.815 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:25.550 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:25.551 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37388/73364 objects degraded (50.962%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":605,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":36682,"data_bytes":150201097,"bytes_used":1736916992,"bytes_avail":288173375488,"bytes_total":289910292480,"degraded_objects":37388,"degraded_total":73364,"degraded_ratio":0.50962324845973506,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:25.574 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:25.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:26.152 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:26.155 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:26.182 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-11T01:23:24.623433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":150201097,"num_objects":36682,"num_object_clones":0,"num_object_copies":73364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36682,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":248563,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12247,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1696208,"kb_used_data":496088,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281419312,"statfs":{"total":289910292480,"available":288173375488,"internally_reserved":0,"allocated":507994112,"data_stored":507108416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1228852231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,5,6,5,13,20,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24313856,"num_objects":-5936,"num_object_clones":0,"num_object_copies":-11872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5936,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003965"},"pg_stats":[{"pgid":"2.f","version":"43'17985","reported_seq":18209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.537581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:23.537581+0000","last_peered":"2024-09-11T01:23:23.537581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:23.537581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1795}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18131","reported_seq":18212,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.053388+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:21.053388+0000","last_peered":"2024-09-11T01:23:21.053388+0000","last_clean":"2024-09-11T01:23:21.053388+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:21.053388+0000","last_undegraded":"2024-09-11T01:23:21.053388+0000","last_fullsized":"2024-09-11T01:23:21.053388+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9794502,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16229,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17778","reported_seq":17946,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.102423+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:21.102423+0000","last_peered":"2024-09-11T01:23:21.102423+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:21.102423+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9285656,"num_objects":2268,"num_object_clones":0,"num_object_copies":4536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2268,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15839,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'17995","reported_seq":18041,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.041834+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:21.041834+0000","last_peered":"2024-09-11T01:23:21.041834+0000","last_clean":"2024-09-11T01:23:21.041834+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:21.041834+0000","last_undegraded":"2024-09-11T01:23:21.041834+0000","last_fullsized":"2024-09-11T01:23:21.041834+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9695232,"num_objects":2367,"num_object_clones":0,"num_object_copies":4734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17995,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17992","reported_seq":18115,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.373606+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:23.373606+0000","last_peered":"2024-09-11T01:23:23.373606+0000","last_clean":"2024-09-11T01:23:23.373606+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:23.373606+0000","last_undegraded":"2024-09-11T01:23:23.373606+0000","last_fullsized":"2024-09-11T01:23:23.373606+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359843,"num_objects":2286,"num_object_clones":0,"num_object_copies":4572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14790,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17999","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.555384+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:23.555384+0000","last_peered":"2024-09-11T01:23:23.555384+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:23.555384+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9044451,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14487,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18509","reported_seq":18797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.542651+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:23.542651+0000","last_peered":"2024-09-11T01:23:23.542651+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:23.542651+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9434054,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15262,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1901},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18160","reported_seq":18281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.514831+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:23.514831+0000","last_peered":"2024-09-11T01:23:23.514831+0000","last_clean":"2024-09-11T01:23:23.514831+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:23.514831+0000","last_undegraded":"2024-09-11T01:23:23.514831+0000","last_fullsized":"2024-09-11T01:23:23.514831+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9273344,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15113,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18000","reported_seq":26344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.700007+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:21.700007+0000","last_peered":"2024-09-11T01:23:21.700007+0000","last_clean":"2024-09-11T01:23:21.700007+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:21.700007+0000","last_undegraded":"2024-09-11T01:23:21.700007+0000","last_fullsized":"2024-09-11T01:23:21.700007+0000","mapping_epoch":40,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9212387,"num_objects":2250,"num_object_clones":0,"num_object_copies":4500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15867,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18310","reported_seq":18586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.516755+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:23.516755+0000","last_peered":"2024-09-11T01:23:23.516755+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:23.516755+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17749","reported_seq":24606,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.140180+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:21.140180+0000","last_peered":"2024-09-11T01:23:21.140180+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:21.140180+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7700","ondisk_log_start":"39'7700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9789923,"num_objects":2391,"num_object_clones":0,"num_object_copies":4782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15518,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3380,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1743}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18252","reported_seq":18406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:21.086072+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:21.086072+0000","last_peered":"2024-09-11T01:23:21.086072+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:21.086072+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9461760,"num_objects":2310,"num_object_clones":0,"num_object_copies":4620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15889,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1837}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18028","reported_seq":26500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:21.581146+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:21.581146+0000","last_peered":"2024-09-11T01:23:21.581146+0000","last_clean":"2024-09-11T01:23:21.581146+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:21.581146+0000","last_undegraded":"2024-09-11T01:23:21.581146+0000","last_fullsized":"2024-09-11T01:23:21.581146+0000","mapping_epoch":42,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14718,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18411","reported_seq":18697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.529197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:23.529197+0000","last_peered":"2024-09-11T01:23:23.529197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:23.529197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18276","reported_seq":18577,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.496017+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:23.496017+0000","last_peered":"2024-09-11T01:23:23.496017+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:23.496017+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14811,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1741}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17923","reported_seq":18162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.566406+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:23.566406+0000","last_peered":"2024-09-11T01:23:23.566406+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:23.566406+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073606,"num_objects":2217,"num_object_clones":0,"num_object_copies":4434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14580,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":150201078,"num_objects":36680,"num_object_clones":0,"num_object_copies":73360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36680,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":248561,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12246,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":504950784,"data_stored":504798120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160898,"ondisk_log_size":160898,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528816,"kb_used_data":208352,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93843024,"statfs":{"total":96636764160,"available":96095256576,"internally_reserved":0,"allocated":213352448,"data_stored":213070620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790049,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":554520,"kb_used_data":111876,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93817320,"statfs":{"total":96636764160,"available":96068935680,"internally_reserved":0,"allocated":114561024,"data_stored":114238989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612872,"kb_used_data":175860,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93758968,"statfs":{"total":96636764160,"available":96009183232,"internally_reserved":0,"allocated":180080640,"data_stored":179798807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178393088,"data_stored":178356958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113623040,"data_stored":113546708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":212934656,"data_stored":212894454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:29.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:29.766 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:29.767 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:29.790 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-11T01:23:28.625244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145994505,"num_objects":35655,"num_object_clones":0,"num_object_copies":71310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35655,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":249590,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12380,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1697128,"kb_used_data":484520,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281418392,"statfs":{"total":289910292480,"available":288172433408,"internally_reserved":0,"allocated":496148480,"data_stored":495262784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1241631751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,5,13,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20492288,"num_objects":-5003,"num_object_clones":0,"num_object_copies":-10006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5003,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004586"},"pg_stats":[{"pgid":"2.f","version":"43'17985","reported_seq":18209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.537581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:23.537581+0000","last_peered":"2024-09-11T01:23:23.537581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:23.537581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1795}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18286","reported_seq":18368,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.143866+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:26.143866+0000","last_peered":"2024-09-11T01:23:26.143866+0000","last_clean":"2024-09-11T01:23:26.143866+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:26.143866+0000","last_undegraded":"2024-09-11T01:23:26.143866+0000","last_fullsized":"2024-09-11T01:23:26.143866+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9159622,"num_objects":2238,"num_object_clones":0,"num_object_copies":4476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2238,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16384,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17932","reported_seq":18101,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.108577+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:26.108577+0000","last_peered":"2024-09-11T01:23:26.108577+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:26.108577+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654872,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15993,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1849}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18150","reported_seq":18197,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.157570+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:26.157570+0000","last_peered":"2024-09-11T01:23:26.157570+0000","last_clean":"2024-09-11T01:23:26.157570+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:26.157570+0000","last_undegraded":"2024-09-11T01:23:26.157570+0000","last_fullsized":"2024-09-11T01:23:26.157570+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9060352,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18150,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17992","reported_seq":18115,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.373606+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:23.373606+0000","last_peered":"2024-09-11T01:23:23.373606+0000","last_clean":"2024-09-11T01:23:23.373606+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:23.373606+0000","last_undegraded":"2024-09-11T01:23:23.373606+0000","last_fullsized":"2024-09-11T01:23:23.373606+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359843,"num_objects":2286,"num_object_clones":0,"num_object_copies":4572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14790,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17999","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.555384+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:23.555384+0000","last_peered":"2024-09-11T01:23:23.555384+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:23.555384+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9044451,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14487,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18509","reported_seq":18797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.542651+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:23.542651+0000","last_peered":"2024-09-11T01:23:23.542651+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:23.542651+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9434054,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15262,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1901},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18160","reported_seq":18281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.514831+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:23.514831+0000","last_peered":"2024-09-11T01:23:23.514831+0000","last_clean":"2024-09-11T01:23:23.514831+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:23.514831+0000","last_undegraded":"2024-09-11T01:23:23.514831+0000","last_fullsized":"2024-09-11T01:23:23.514831+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9273344,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15113,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18138","reported_seq":26482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.759750+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:26.759750+0000","last_peered":"2024-09-11T01:23:26.759750+0000","last_clean":"2024-09-11T01:23:26.759750+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:26.759750+0000","last_undegraded":"2024-09-11T01:23:26.759750+0000","last_fullsized":"2024-09-11T01:23:26.759750+0000","mapping_epoch":40,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8647139,"num_objects":2112,"num_object_clones":0,"num_object_copies":4224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18310","reported_seq":18586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.516755+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:23.516755+0000","last_peered":"2024-09-11T01:23:23.516755+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:23.516755+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17891","reported_seq":25012,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.148285+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:26.148285+0000","last_peered":"2024-09-11T01:23:26.148285+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:26.148285+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9208291,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15660,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3513,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18376","reported_seq":18531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.089539+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:26.089539+0000","last_peered":"2024-09-11T01:23:26.089539+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:26.089539+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18187","reported_seq":26659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.708940+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:26.708940+0000","last_peered":"2024-09-11T01:23:26.708940+0000","last_clean":"2024-09-11T01:23:26.708940+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:26.708940+0000","last_undegraded":"2024-09-11T01:23:26.708940+0000","last_fullsized":"2024-09-11T01:23:26.708940+0000","mapping_epoch":42,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14877,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18411","reported_seq":18697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.529197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:23.529197+0000","last_peered":"2024-09-11T01:23:23.529197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:23.529197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18276","reported_seq":18577,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.496017+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:23.496017+0000","last_peered":"2024-09-11T01:23:23.496017+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:23.496017+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14811,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1741}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17923","reported_seq":18162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.566406+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:23.566406+0000","last_peered":"2024-09-11T01:23:23.566406+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:23.566406+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073606,"num_objects":2217,"num_object_clones":0,"num_object_copies":4434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14580,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145994486,"num_objects":35653,"num_object_clones":0,"num_object_copies":71306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35653,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":249588,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12379,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":492974080,"data_stored":492821416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528332,"kb_used_data":203708,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93843508,"statfs":{"total":96636764160,"available":96095752192,"internally_reserved":0,"allocated":208596992,"data_stored":208315164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790050,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555924,"kb_used_data":104952,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93815916,"statfs":{"total":96636764160,"available":96067497984,"internally_reserved":0,"allocated":107470848,"data_stored":107148813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612872,"kb_used_data":175860,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93758968,"statfs":{"total":96636764160,"available":96009183232,"internally_reserved":0,"allocated":180080640,"data_stored":179798807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178393088,"data_stored":178356958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106389504,"data_stored":106313172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208191488,"data_stored":208151286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:29.792 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:30.381 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:30.382 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:30.407 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-11T01:23:28.625244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145994505,"num_objects":35655,"num_object_clones":0,"num_object_copies":71310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35655,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":249590,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12380,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1697128,"kb_used_data":484520,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281418392,"statfs":{"total":289910292480,"available":288172433408,"internally_reserved":0,"allocated":496148480,"data_stored":495262784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1241631751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,5,13,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20492288,"num_objects":-5003,"num_object_clones":0,"num_object_copies":-10006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5003,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004586"},"pg_stats":[{"pgid":"2.f","version":"43'17985","reported_seq":18209,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.537581+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:23.537581+0000","last_peered":"2024-09-11T01:23:23.537581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:23.537581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9228288,"num_objects":2253,"num_object_clones":0,"num_object_copies":4506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17985,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1795}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18286","reported_seq":18368,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.143866+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:26.143866+0000","last_peered":"2024-09-11T01:23:26.143866+0000","last_clean":"2024-09-11T01:23:26.143866+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:26.143866+0000","last_undegraded":"2024-09-11T01:23:26.143866+0000","last_fullsized":"2024-09-11T01:23:26.143866+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9159622,"num_objects":2238,"num_object_clones":0,"num_object_copies":4476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2238,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16384,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17932","reported_seq":18101,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.108577+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:26.108577+0000","last_peered":"2024-09-11T01:23:26.108577+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:26.108577+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654872,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15993,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1849}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18150","reported_seq":18197,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.157570+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:26.157570+0000","last_peered":"2024-09-11T01:23:26.157570+0000","last_clean":"2024-09-11T01:23:26.157570+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:26.157570+0000","last_undegraded":"2024-09-11T01:23:26.157570+0000","last_fullsized":"2024-09-11T01:23:26.157570+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9060352,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18150,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'17992","reported_seq":18115,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.373606+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:23.373606+0000","last_peered":"2024-09-11T01:23:23.373606+0000","last_clean":"2024-09-11T01:23:23.373606+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:23.373606+0000","last_undegraded":"2024-09-11T01:23:23.373606+0000","last_fullsized":"2024-09-11T01:23:23.373606+0000","mapping_epoch":31,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359843,"num_objects":2286,"num_object_clones":0,"num_object_copies":4572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2286,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14790,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'17999","reported_seq":18266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.555384+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:23.555384+0000","last_peered":"2024-09-11T01:23:23.555384+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:23.555384+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9044451,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14487,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18509","reported_seq":18797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.542651+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:23.542651+0000","last_peered":"2024-09-11T01:23:23.542651+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:23.542651+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9434054,"num_objects":2305,"num_object_clones":0,"num_object_copies":4610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2305,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15262,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1901},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18160","reported_seq":18281,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:23.514831+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:23.514831+0000","last_peered":"2024-09-11T01:23:23.514831+0000","last_clean":"2024-09-11T01:23:23.514831+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:23.514831+0000","last_undegraded":"2024-09-11T01:23:23.514831+0000","last_fullsized":"2024-09-11T01:23:23.514831+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9273344,"num_objects":2264,"num_object_clones":0,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15113,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18138","reported_seq":26482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.759750+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:26.759750+0000","last_peered":"2024-09-11T01:23:26.759750+0000","last_clean":"2024-09-11T01:23:26.759750+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:26.759750+0000","last_undegraded":"2024-09-11T01:23:26.759750+0000","last_fullsized":"2024-09-11T01:23:26.759750+0000","mapping_epoch":40,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8647139,"num_objects":2112,"num_object_clones":0,"num_object_copies":4224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18310","reported_seq":18586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.516755+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:23.516755+0000","last_peered":"2024-09-11T01:23:23.516755+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:23.516755+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9355264,"num_objects":2284,"num_object_clones":0,"num_object_copies":4568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14674,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1746}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17891","reported_seq":25012,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.148285+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:26.148285+0000","last_peered":"2024-09-11T01:23:26.148285+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:26.148285+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9208291,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15660,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3513,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18376","reported_seq":18531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.089539+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:26.089539+0000","last_peered":"2024-09-11T01:23:26.089539+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:26.089539+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18187","reported_seq":26659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.708940+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:26.708940+0000","last_peered":"2024-09-11T01:23:26.708940+0000","last_clean":"2024-09-11T01:23:26.708940+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:26.708940+0000","last_undegraded":"2024-09-11T01:23:26.708940+0000","last_fullsized":"2024-09-11T01:23:26.708940+0000","mapping_epoch":42,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14877,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18411","reported_seq":18697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.529197+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:23.529197+0000","last_peered":"2024-09-11T01:23:23.529197+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:23.529197+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9154560,"num_objects":2235,"num_object_clones":0,"num_object_copies":4470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1882}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18276","reported_seq":18577,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.496017+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:23.496017+0000","last_peered":"2024-09-11T01:23:23.496017+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:23.496017+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14811,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1741}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'17923","reported_seq":18162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.566406+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:23.566406+0000","last_peered":"2024-09-11T01:23:23.566406+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:23.566406+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073606,"num_objects":2217,"num_object_clones":0,"num_object_copies":4434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2217,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14580,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145994486,"num_objects":35653,"num_object_clones":0,"num_object_copies":71306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35653,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":249588,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12379,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":492974080,"data_stored":492821416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528332,"kb_used_data":203708,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93843508,"statfs":{"total":96636764160,"available":96095752192,"internally_reserved":0,"allocated":208596992,"data_stored":208315164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790050,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555924,"kb_used_data":104952,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93815916,"statfs":{"total":96636764160,"available":96067497984,"internally_reserved":0,"allocated":107470848,"data_stored":107148813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612872,"kb_used_data":175860,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93758968,"statfs":{"total":96636764160,"available":96009183232,"internally_reserved":0,"allocated":180080640,"data_stored":179798807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,1,1,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178393088,"data_stored":178356958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106389504,"data_stored":106313172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208191488,"data_stored":208151286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'17985", 'reported_seq': 18209, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.537581+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:23.537581+0000', 'last_peered': '2024-09-11T01:23:23.537581+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:23.537581+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9228288, 'num_objects': 2253, 'num_object_clones': 0, 'num_object_copies': 4506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2253, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17985, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1795}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'17932", 'reported_seq': 18101, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:26.108577+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:26.108577+0000', 'last_peered': '2024-09-11T01:23:26.108577+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:26.108577+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8654872, 'num_objects': 2114, 'num_object_clones': 0, 'num_object_copies': 4228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2114, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15993, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1849}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'17999", 'reported_seq': 18266, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.555384+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:23.555384+0000', 'last_peered': '2024-09-11T01:23:23.555384+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:23.555384+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9044451, 'num_objects': 2209, 'num_object_clones': 0, 'num_object_copies': 4418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2209, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14487, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1840}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'18509", 'reported_seq': 18797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.542651+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:23.542651+0000', 'last_peered': '2024-09-11T01:23:23.542651+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:23.542651+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'8500", 'ondisk_log_start': "39'8500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9434054, 'num_objects': 2305, 'num_object_clones': 0, 'num_object_copies': 4610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2305, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15262, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1901}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.409 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'18310", 'reported_seq': 18586, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.516755+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:23.516755+0000', 'last_peered': '2024-09-11T01:23:23.516755+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:23.516755+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'8300", 'ondisk_log_start': "39'8300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9355264, 'num_objects': 2284, 'num_object_clones': 0, 'num_object_copies': 4568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2284, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14674, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1746}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'17891", 'reported_seq': 25012, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:26.148285+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:26.148285+0000', 'last_peered': '2024-09-11T01:23:26.148285+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:26.148285+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'7800", 'ondisk_log_start': "39'7800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9208291, 'num_objects': 2249, 'num_object_clones': 0, 'num_object_copies': 4498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 515, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2249, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15660, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3513, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1734}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'18376", 'reported_seq': 18531, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:26.089539+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:26.089539+0000', 'last_peered': '2024-09-11T01:23:26.089539+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:26.089539+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'8300", 'ondisk_log_start': "39'8300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8953856, 'num_objects': 2186, 'num_object_clones': 0, 'num_object_copies': 4372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16013, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1961}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'18411", 'reported_seq': 18697, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.529197+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:23.529197+0000', 'last_peered': '2024-09-11T01:23:23.529197+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:23.529197+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'8400", 'ondisk_log_start': "39'8400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9154560, 'num_objects': 2235, 'num_object_clones': 0, 'num_object_copies': 4470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14804, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1882}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'18276", 'reported_seq': 18577, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.496017+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:23.496017+0000', 'last_peered': '2024-09-11T01:23:23.496017+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:23.496017+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9519104, 'num_objects': 2324, 'num_object_clones': 0, 'num_object_copies': 4648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14811, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:30.410 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'17923", 'reported_seq': 18162, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.566406+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:23.566406+0000', 'last_peered': '2024-09-11T01:23:23.566406+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:23.566406+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'7900", 'ondisk_log_start': "39'7900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9073606, 'num_objects': 2217, 'num_object_clones': 0, 'num_object_copies': 4434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2217, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14580, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1885}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:30.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:31.111 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:31.111 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37255/68690 objects degraded (54.236%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":611,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":34345,"data_bytes":140628745,"bytes_used":1717936128,"bytes_avail":288192356352,"bytes_total":289910292480,"degraded_objects":37255,"degraded_total":68690,"degraded_ratio":0.54236424515941184,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:31.140 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:31.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:31.711 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:31.712 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:31.738 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-11T01:23:30.626130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":140628745,"num_objects":34345,"num_object_clones":0,"num_object_copies":68690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34345,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":250900,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12380,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1677672,"kb_used_data":477548,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281437848,"statfs":{"total":289910292480,"available":288192356352,"internally_reserved":0,"allocated":489009152,"data_stored":488123456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1228852231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,6,10,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19181568,"num_objects":-4683,"num_object_clones":0,"num_object_copies":-9366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4683,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004852"},"pg_stats":[{"pgid":"2.f","version":"43'18124","reported_seq":18349,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.588794+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:28.588794+0000","last_peered":"2024-09-11T01:23:28.588794+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:28.588794+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8658944,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18124,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1934}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18286","reported_seq":18368,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.143866+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:26.143866+0000","last_peered":"2024-09-11T01:23:26.143866+0000","last_clean":"2024-09-11T01:23:26.143866+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:26.143866+0000","last_undegraded":"2024-09-11T01:23:26.143866+0000","last_fullsized":"2024-09-11T01:23:26.143866+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9159622,"num_objects":2238,"num_object_clones":0,"num_object_copies":4476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2238,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16384,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'17932","reported_seq":18101,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.108577+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:26.108577+0000","last_peered":"2024-09-11T01:23:26.108577+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:26.108577+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'7900","ondisk_log_start":"39'7900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8654872,"num_objects":2114,"num_object_clones":0,"num_object_copies":4228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15993,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1849}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18150","reported_seq":18197,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.157570+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:26.157570+0000","last_peered":"2024-09-11T01:23:26.157570+0000","last_clean":"2024-09-11T01:23:26.157570+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:26.157570+0000","last_undegraded":"2024-09-11T01:23:26.157570+0000","last_fullsized":"2024-09-11T01:23:26.157570+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9060352,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18150,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18150","reported_seq":18273,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:28.588673+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:28.588673+0000","last_peered":"2024-09-11T01:23:28.588673+0000","last_clean":"2024-09-11T01:23:28.588673+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:28.588673+0000","last_undegraded":"2024-09-11T01:23:28.588673+0000","last_fullsized":"2024-09-11T01:23:28.588673+0000","mapping_epoch":31,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712675,"num_objects":2128,"num_object_clones":0,"num_object_copies":4256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2128,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14948,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18171","reported_seq":18439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.558215+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:28.558215+0000","last_peered":"2024-09-11T01:23:28.558215+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:28.558215+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8339939,"num_objects":2037,"num_object_clones":0,"num_object_copies":4074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2037,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14659,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2012},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18643","reported_seq":18932,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.570493+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:28.570493+0000","last_peered":"2024-09-11T01:23:28.570493+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:28.570493+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8885190,"num_objects":2171,"num_object_clones":0,"num_object_copies":4342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2171,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15396,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2035},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18297","reported_seq":18415,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:28.619091+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:28.619091+0000","last_peered":"2024-09-11T01:23:28.619091+0000","last_clean":"2024-09-11T01:23:28.619091+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:28.619091+0000","last_undegraded":"2024-09-11T01:23:28.619091+0000","last_fullsized":"2024-09-11T01:23:28.619091+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8712192,"num_objects":2127,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15250,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18138","reported_seq":26482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.759750+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:26.759750+0000","last_peered":"2024-09-11T01:23:26.759750+0000","last_clean":"2024-09-11T01:23:26.759750+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:26.759750+0000","last_undegraded":"2024-09-11T01:23:26.759750+0000","last_fullsized":"2024-09-11T01:23:26.759750+0000","mapping_epoch":40,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8647139,"num_objects":2112,"num_object_clones":0,"num_object_copies":4224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16005,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18459","reported_seq":18739,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.597300+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:28.597300+0000","last_peered":"2024-09-11T01:23:28.597300+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:28.597300+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744960,"num_objects":2135,"num_object_clones":0,"num_object_copies":4270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14823,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'17891","reported_seq":25012,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.148285+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:26.148285+0000","last_peered":"2024-09-11T01:23:26.148285+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:26.148285+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'7800","ondisk_log_start":"39'7800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9208291,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":515,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15660,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3513,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1734}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18376","reported_seq":18531,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:26.089539+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:26.089539+0000","last_peered":"2024-09-11T01:23:26.089539+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:26.089539+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16013,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1961}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18187","reported_seq":26659,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:26.708940+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:26.708940+0000","last_peered":"2024-09-11T01:23:26.708940+0000","last_clean":"2024-09-11T01:23:26.708940+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:26.708940+0000","last_undegraded":"2024-09-11T01:23:26.708940+0000","last_fullsized":"2024-09-11T01:23:26.708940+0000","mapping_epoch":42,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8867840,"num_objects":2165,"num_object_clones":0,"num_object_copies":4330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14877,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18552","reported_seq":18838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.620999+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:28.620999+0000","last_peered":"2024-09-11T01:23:28.620999+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:28.620999+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8577024,"num_objects":2094,"num_object_clones":0,"num_object_copies":4188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14945,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18416","reported_seq":18713,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.563314+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:28.563314+0000","last_peered":"2024-09-11T01:23:28.563314+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:28.563314+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8945664,"num_objects":2184,"num_object_clones":0,"num_object_copies":4368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14951,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1881}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18063","reported_seq":18303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.499565+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:28.499565+0000","last_peered":"2024-09-11T01:23:28.499565+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:28.499565+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8500166,"num_objects":2077,"num_object_clones":0,"num_object_copies":4154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2077,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14720,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2025},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":140628726,"num_objects":34343,"num_object_clones":0,"num_object_copies":68686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37255,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34343,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":250898,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12379,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":485859328,"data_stored":485706664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528332,"kb_used_data":203708,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93843508,"statfs":{"total":96636764160,"available":96095752192,"internally_reserved":0,"allocated":208596992,"data_stored":208315164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790050,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555924,"kb_used_data":104952,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93815916,"statfs":{"total":96636764160,"available":96067497984,"internally_reserved":0,"allocated":107470848,"data_stored":107148813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593416,"kb_used_data":168888,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93778424,"statfs":{"total":96636764160,"available":96029106176,"internally_reserved":0,"allocated":172941312,"data_stored":172659479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171278336,"data_stored":171242206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":106389504,"data_stored":106313172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":208191488,"data_stored":208151286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:34.740 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:36.061 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:36.061 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:36.088 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-11T01:23:34.627864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":131302153,"num_objects":32068,"num_object_clones":0,"num_object_copies":64136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32068,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":253177,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12488,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1671456,"kb_used_data":458848,"kb_used_omap":12,"kb_used_meta":1212531,"kb_avail":281444064,"statfs":{"total":289910292480,"available":288198721536,"internally_reserved":0,"allocated":469860352,"data_stored":468974656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1241631751},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,10,9,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24395776,"num_objects":-5956,"num_object_clones":0,"num_object_copies":-11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-241,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5956,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005332"},"pg_stats":[{"pgid":"2.f","version":"43'18238","reported_seq":18463,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.641614+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:33.641614+0000","last_peered":"2024-09-11T01:23:33.641614+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:33.641614+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18423","reported_seq":18505,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.153972+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:31.153972+0000","last_peered":"2024-09-11T01:23:31.153972+0000","last_clean":"2024-09-11T01:23:31.153972+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:31.153972+0000","last_undegraded":"2024-09-11T01:23:31.153972+0000","last_fullsized":"2024-09-11T01:23:31.153972+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8598470,"num_objects":2101,"num_object_clones":0,"num_object_copies":4202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2101,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16521,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'18068","reported_seq":18236,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:31.117151+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:31.117151+0000","last_peered":"2024-09-11T01:23:31.117151+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:31.117151+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097816,"num_objects":1978,"num_object_clones":0,"num_object_copies":3956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1978,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16129,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1985}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18296","reported_seq":18343,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.160381+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:31.160381+0000","last_peered":"2024-09-11T01:23:31.160381+0000","last_clean":"2024-09-11T01:23:31.160381+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:31.160381+0000","last_undegraded":"2024-09-11T01:23:31.160381+0000","last_fullsized":"2024-09-11T01:23:31.160381+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8462336,"num_objects":2066,"num_object_clones":0,"num_object_copies":4132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18296,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18295","reported_seq":18417,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:33.590378+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:33.590378+0000","last_peered":"2024-09-11T01:23:33.590378+0000","last_clean":"2024-09-11T01:23:33.590378+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:33.590378+0000","last_undegraded":"2024-09-11T01:23:33.590378+0000","last_fullsized":"2024-09-11T01:23:33.590378+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8118755,"num_objects":1983,"num_object_clones":0,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15093,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18294","reported_seq":18562,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.619147+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:33.619147+0000","last_peered":"2024-09-11T01:23:33.619147+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:33.619147+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7836131,"num_objects":1914,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14782,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2135},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18786","reported_seq":19075,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.613576+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:33.613576+0000","last_peered":"2024-09-11T01:23:33.613576+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:33.613576+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8700","ondisk_log_start":"39'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8299462,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15539,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2178},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18420","reported_seq":18541,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:33.587574+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:33.587574+0000","last_peered":"2024-09-11T01:23:33.587574+0000","last_clean":"2024-09-11T01:23:33.587574+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:33.587574+0000","last_undegraded":"2024-09-11T01:23:33.587574+0000","last_fullsized":"2024-09-11T01:23:33.587574+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8208384,"num_objects":2004,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15373,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18292","reported_seq":26636,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.749143+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:31.749143+0000","last_peered":"2024-09-11T01:23:31.749143+0000","last_clean":"2024-09-11T01:23:31.749143+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:31.749143+0000","last_undegraded":"2024-09-11T01:23:31.749143+0000","last_fullsized":"2024-09-11T01:23:31.749143+0000","mapping_epoch":40,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8016355,"num_objects":1958,"num_object_clones":0,"num_object_copies":3916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16159,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18601","reported_seq":18882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.615491+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:33.615491+0000","last_peered":"2024-09-11T01:23:33.615491+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:33.615491+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14965,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2037}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18061","reported_seq":25400,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:31.125625+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:31.125625+0000","last_peered":"2024-09-11T01:23:31.125625+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:31.125625+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'8000","ondisk_log_start":"39'8000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8511971,"num_objects":2079,"num_object_clones":0,"num_object_copies":4158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2079,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15830,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3621,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1672}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18530","reported_seq":18685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:31.143304+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:31.143304+0000","last_peered":"2024-09-11T01:23:31.143304+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:31.143304+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8323072,"num_objects":2032,"num_object_clones":0,"num_object_copies":4064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16167,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18330","reported_seq":26802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.762651+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:31.762651+0000","last_peered":"2024-09-11T01:23:31.762651+0000","last_clean":"2024-09-11T01:23:31.762651+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:31.762651+0000","last_undegraded":"2024-09-11T01:23:31.762651+0000","last_fullsized":"2024-09-11T01:23:31.762651+0000","mapping_epoch":42,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8282112,"num_objects":2022,"num_object_clones":0,"num_object_copies":4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18716","reported_seq":19001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.589623+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:33.589623+0000","last_peered":"2024-09-11T01:23:33.589623+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:33.589623+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8700","ondisk_log_start":"39'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7905280,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2187}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18549","reported_seq":18850,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.601787+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:33.601787+0000","last_peered":"2024-09-11T01:23:33.601787+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:33.601787+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18213","reported_seq":18451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.618635+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:33.618635+0000","last_peered":"2024-09-11T01:23:33.618635+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:33.618635+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7885766,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14870,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2175},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":131302134,"num_objects":32066,"num_object_clones":0,"num_object_copies":64132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37147,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32066,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":253175,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12487,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466931712,"data_stored":466779048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527736,"kb_used_data":198948,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93844104,"statfs":{"total":96636764160,"available":96096362496,"internally_reserved":0,"allocated":203722752,"data_stored":203440924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,3,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790051,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552896,"kb_used_data":97768,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93818944,"statfs":{"total":96636764160,"available":96070598656,"internally_reserved":0,"allocated":100114432,"data_stored":99792397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590824,"kb_used_data":162132,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93781016,"statfs":{"total":96636764160,"available":96031760384,"internally_reserved":0,"allocated":166023168,"data_stored":165741335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164438016,"data_stored":164401886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":99221504,"data_stored":99145172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203272192,"data_stored":203231990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:36.090 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:36.642 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:36.644 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:36.675 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-11T01:23:36.628665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128488201,"num_objects":31381,"num_object_clones":0,"num_object_copies":62762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31381,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":253864,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12656,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1654500,"kb_used_data":454376,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281461020,"statfs":{"total":289910292480,"available":288216084480,"internally_reserved":0,"allocated":465281024,"data_stored":464395328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1228852231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,9,10,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21712896,"num_objects":-5301,"num_object_clones":0,"num_object_copies":-10602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5301,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005231"},"pg_stats":[{"pgid":"2.f","version":"43'18238","reported_seq":18463,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.641614+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:33.641614+0000","last_peered":"2024-09-11T01:23:33.641614+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:33.641614+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18562","reported_seq":18644,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.000002+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:36.000002+0000","last_peered":"2024-09-11T01:23:36.000002+0000","last_clean":"2024-09-11T01:23:36.000002+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:36.000002+0000","last_undegraded":"2024-09-11T01:23:36.000002+0000","last_fullsized":"2024-09-11T01:23:36.000002+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8029126,"num_objects":1962,"num_object_clones":0,"num_object_copies":3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16660,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'18204","reported_seq":18368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.088377+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:36.088377+0000","last_peered":"2024-09-11T01:23:36.088377+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:36.088377+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7540760,"num_objects":1842,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16265,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18419","reported_seq":18466,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.090146+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:36.090146+0000","last_peered":"2024-09-11T01:23:36.090146+0000","last_clean":"2024-09-11T01:23:36.090146+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:36.090146+0000","last_undegraded":"2024-09-11T01:23:36.090146+0000","last_fullsized":"2024-09-11T01:23:36.090146+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7958528,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18419,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18295","reported_seq":18417,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:33.590378+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:33.590378+0000","last_peered":"2024-09-11T01:23:33.590378+0000","last_clean":"2024-09-11T01:23:33.590378+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:33.590378+0000","last_undegraded":"2024-09-11T01:23:33.590378+0000","last_fullsized":"2024-09-11T01:23:33.590378+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8118755,"num_objects":1983,"num_object_clones":0,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15093,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18294","reported_seq":18562,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.619147+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:33.619147+0000","last_peered":"2024-09-11T01:23:33.619147+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:33.619147+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7836131,"num_objects":1914,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14782,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2135},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18786","reported_seq":19075,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.613576+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:33.613576+0000","last_peered":"2024-09-11T01:23:33.613576+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:33.613576+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8700","ondisk_log_start":"39'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8299462,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15539,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2178},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18420","reported_seq":18541,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:33.587574+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:33.587574+0000","last_peered":"2024-09-11T01:23:33.587574+0000","last_clean":"2024-09-11T01:23:33.587574+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:33.587574+0000","last_undegraded":"2024-09-11T01:23:33.587574+0000","last_fullsized":"2024-09-11T01:23:33.587574+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8208384,"num_objects":2004,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15373,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18292","reported_seq":26636,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.749143+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:31.749143+0000","last_peered":"2024-09-11T01:23:31.749143+0000","last_clean":"2024-09-11T01:23:31.749143+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:31.749143+0000","last_undegraded":"2024-09-11T01:23:31.749143+0000","last_fullsized":"2024-09-11T01:23:31.749143+0000","mapping_epoch":40,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8016355,"num_objects":1958,"num_object_clones":0,"num_object_copies":3916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16159,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18601","reported_seq":18882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.615491+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:33.615491+0000","last_peered":"2024-09-11T01:23:33.615491+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:33.615491+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14965,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2037}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18197","reported_seq":25872,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.165956+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:36.165956+0000","last_peered":"2024-09-11T01:23:36.165956+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:36.165956+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7954915,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15966,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3789,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18683","reported_seq":18837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.179408+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:36.179408+0000","last_peered":"2024-09-11T01:23:36.179408+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:36.179408+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16320,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2268}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18330","reported_seq":26802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.762651+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:31.762651+0000","last_peered":"2024-09-11T01:23:31.762651+0000","last_clean":"2024-09-11T01:23:31.762651+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:31.762651+0000","last_undegraded":"2024-09-11T01:23:31.762651+0000","last_fullsized":"2024-09-11T01:23:31.762651+0000","mapping_epoch":42,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8282112,"num_objects":2022,"num_object_clones":0,"num_object_copies":4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18716","reported_seq":19001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.589623+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:33.589623+0000","last_peered":"2024-09-11T01:23:33.589623+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:33.589623+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8700","ondisk_log_start":"39'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7905280,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2187}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18549","reported_seq":18850,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.601787+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:33.601787+0000","last_peered":"2024-09-11T01:23:33.601787+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:33.601787+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18213","reported_seq":18451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.618635+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:33.618635+0000","last_peered":"2024-09-11T01:23:33.618635+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:33.618635+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7885766,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14870,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2175},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128488182,"num_objects":31379,"num_object_clones":0,"num_object_copies":62758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31379,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":253862,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12655,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462385152,"data_stored":462232488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510780,"kb_used_data":194476,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93861060,"statfs":{"total":96636764160,"available":96113725440,"internally_reserved":0,"allocated":199143424,"data_stored":198861596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790051,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552896,"kb_used_data":97768,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93818944,"statfs":{"total":96636764160,"available":96070598656,"internally_reserved":0,"allocated":100114432,"data_stored":99792397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590824,"kb_used_data":162132,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93781016,"statfs":{"total":96636764160,"available":96031760384,"internally_reserved":0,"allocated":166023168,"data_stored":165741335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164438016,"data_stored":164401886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":99221504,"data_stored":99145172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198725632,"data_stored":198685430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:36.677 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:36.677 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'18238", 'reported_seq': 18463, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.641614+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:33.641614+0000', 'last_peered': '2024-09-11T01:23:33.641614+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:33.641614+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8192000, 'num_objects': 2000, 'num_object_clones': 0, 'num_object_copies': 4000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2000, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18238, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2048}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.677 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:36.677 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'18204", 'reported_seq': 18368, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:36.088377+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:36.088377+0000', 'last_peered': '2024-09-11T01:23:36.088377+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:36.088377+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7540760, 'num_objects': 1842, 'num_object_clones': 0, 'num_object_copies': 3684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1842, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16265, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:36.677 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:36.677 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'18294", 'reported_seq': 18562, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.619147+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:33.619147+0000', 'last_peered': '2024-09-11T01:23:33.619147+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:33.619147+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7836131, 'num_objects': 1914, 'num_object_clones': 0, 'num_object_copies': 3828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1914, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14782, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2135}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'18786", 'reported_seq': 19075, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.613576+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:33.613576+0000', 'last_peered': '2024-09-11T01:23:33.613576+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:33.613576+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "39'8700", 'ondisk_log_start': "39'8700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8299462, 'num_objects': 2028, 'num_object_clones': 0, 'num_object_copies': 4056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2028, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15539, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2178}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'18601", 'reported_seq': 18882, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.615491+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:33.615491+0000', 'last_peered': '2024-09-11T01:23:33.615491+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:33.615491+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "39'8600", 'ondisk_log_start': "39'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8163328, 'num_objects': 1993, 'num_object_clones': 0, 'num_object_copies': 3986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14965, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2037}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'18197", 'reported_seq': 25872, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:36.165956+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:36.165956+0000', 'last_peered': '2024-09-11T01:23:36.165956+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:36.165956+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'8100", 'ondisk_log_start': "39'8100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7954915, 'num_objects': 1943, 'num_object_clones': 0, 'num_object_copies': 3886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 239, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1943, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15966, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3789, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1704}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'18683", 'reported_seq': 18837, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:36.179408+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:36.179408+0000', 'last_peered': '2024-09-11T01:23:36.179408+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:36.179408+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'8600", 'ondisk_log_start': "39'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7696384, 'num_objects': 1879, 'num_object_clones': 0, 'num_object_copies': 3758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16320, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2268}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:36.678 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'18716", 'reported_seq': 19001, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.589623+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:33.589623+0000', 'last_peered': '2024-09-11T01:23:33.589623+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:33.589623+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "39'8700", 'ondisk_log_start': "39'8700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7905280, 'num_objects': 1930, 'num_object_clones': 0, 'num_object_copies': 3860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15109, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2187}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:36.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'18549", 'reported_seq': 18850, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.601787+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:33.601787+0000', 'last_peered': '2024-09-11T01:23:33.601787+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:33.601787+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'8500", 'ondisk_log_start': "39'8500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8400896, 'num_objects': 2051, 'num_object_clones': 0, 'num_object_copies': 4102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15084, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2014}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.679 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:36.679 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'18213", 'reported_seq': 18451, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.618635+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:33.618635+0000', 'last_peered': '2024-09-11T01:23:33.618635+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:33.618635+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7885766, 'num_objects': 1927, 'num_object_clones': 0, 'num_object_copies': 3854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1927, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14870, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2175}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:36.679 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:37.403 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:37.403 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36979/62762 objects degraded (58.919%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":617,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":31381,"data_bytes":128488201,"bytes_used":1694208000,"bytes_avail":288216084480,"bytes_total":289910292480,"degraded_objects":36979,"degraded_total":62762,"degraded_ratio":0.58919409833975978,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":409,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":441},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:37.434 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:37.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:38.017 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:38.020 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:38.041 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-09-11T01:23:36.628665+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128488201,"num_objects":31381,"num_object_clones":0,"num_object_copies":62762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31381,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":253864,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12656,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1654500,"kb_used_data":454376,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281461020,"statfs":{"total":289910292480,"available":288216084480,"internally_reserved":0,"allocated":465281024,"data_stored":464395328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1228852231},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,9,10,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21712896,"num_objects":-5301,"num_object_clones":0,"num_object_copies":-10602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5301,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005231"},"pg_stats":[{"pgid":"2.f","version":"43'18238","reported_seq":18463,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.641614+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:33.641614+0000","last_peered":"2024-09-11T01:23:33.641614+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:33.641614+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8192000,"num_objects":2000,"num_object_clones":0,"num_object_copies":4000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18562","reported_seq":18644,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.000002+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:36.000002+0000","last_peered":"2024-09-11T01:23:36.000002+0000","last_clean":"2024-09-11T01:23:36.000002+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:36.000002+0000","last_undegraded":"2024-09-11T01:23:36.000002+0000","last_fullsized":"2024-09-11T01:23:36.000002+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8029126,"num_objects":1962,"num_object_clones":0,"num_object_copies":3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16660,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'18204","reported_seq":18368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.088377+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:36.088377+0000","last_peered":"2024-09-11T01:23:36.088377+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:36.088377+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7540760,"num_objects":1842,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16265,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18419","reported_seq":18466,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.090146+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:36.090146+0000","last_peered":"2024-09-11T01:23:36.090146+0000","last_clean":"2024-09-11T01:23:36.090146+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:36.090146+0000","last_undegraded":"2024-09-11T01:23:36.090146+0000","last_fullsized":"2024-09-11T01:23:36.090146+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7958528,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18419,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18295","reported_seq":18417,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:33.590378+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:33.590378+0000","last_peered":"2024-09-11T01:23:33.590378+0000","last_clean":"2024-09-11T01:23:33.590378+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:33.590378+0000","last_undegraded":"2024-09-11T01:23:33.590378+0000","last_fullsized":"2024-09-11T01:23:33.590378+0000","mapping_epoch":31,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8118755,"num_objects":1983,"num_object_clones":0,"num_object_copies":3966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15093,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18294","reported_seq":18562,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.619147+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:33.619147+0000","last_peered":"2024-09-11T01:23:33.619147+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:33.619147+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7836131,"num_objects":1914,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14782,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2135},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18786","reported_seq":19075,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.613576+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:33.613576+0000","last_peered":"2024-09-11T01:23:33.613576+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:33.613576+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"39'8700","ondisk_log_start":"39'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8299462,"num_objects":2028,"num_object_clones":0,"num_object_copies":4056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2028,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15539,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2178},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18420","reported_seq":18541,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:33.587574+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:33.587574+0000","last_peered":"2024-09-11T01:23:33.587574+0000","last_clean":"2024-09-11T01:23:33.587574+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:33.587574+0000","last_undegraded":"2024-09-11T01:23:33.587574+0000","last_fullsized":"2024-09-11T01:23:33.587574+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8208384,"num_objects":2004,"num_object_clones":0,"num_object_copies":4008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15373,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18292","reported_seq":26636,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.749143+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:31.749143+0000","last_peered":"2024-09-11T01:23:31.749143+0000","last_clean":"2024-09-11T01:23:31.749143+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:31.749143+0000","last_undegraded":"2024-09-11T01:23:31.749143+0000","last_fullsized":"2024-09-11T01:23:31.749143+0000","mapping_epoch":40,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8016355,"num_objects":1958,"num_object_clones":0,"num_object_copies":3916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16159,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18601","reported_seq":18882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.615491+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:33.615491+0000","last_peered":"2024-09-11T01:23:33.615491+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:33.615491+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14965,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2037}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18197","reported_seq":25872,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.165956+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:36.165956+0000","last_peered":"2024-09-11T01:23:36.165956+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:36.165956+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7954915,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15966,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3789,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18683","reported_seq":18837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.179408+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:36.179408+0000","last_peered":"2024-09-11T01:23:36.179408+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:36.179408+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16320,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2268}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18330","reported_seq":26802,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:31.762651+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:31.762651+0000","last_peered":"2024-09-11T01:23:31.762651+0000","last_clean":"2024-09-11T01:23:31.762651+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:31.762651+0000","last_undegraded":"2024-09-11T01:23:31.762651+0000","last_fullsized":"2024-09-11T01:23:31.762651+0000","mapping_epoch":42,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8282112,"num_objects":2022,"num_object_clones":0,"num_object_copies":4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18716","reported_seq":19001,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.589623+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:33.589623+0000","last_peered":"2024-09-11T01:23:33.589623+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:33.589623+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"39'8700","ondisk_log_start":"39'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7905280,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2187}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18549","reported_seq":18850,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.601787+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:33.601787+0000","last_peered":"2024-09-11T01:23:33.601787+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:33.601787+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8400896,"num_objects":2051,"num_object_clones":0,"num_object_copies":4102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15084,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2014}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18213","reported_seq":18451,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.618635+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:33.618635+0000","last_peered":"2024-09-11T01:23:33.618635+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:33.618635+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7885766,"num_objects":1927,"num_object_clones":0,"num_object_copies":3854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14870,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2175},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128488182,"num_objects":31379,"num_object_clones":0,"num_object_copies":62758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31379,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":253862,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12655,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462385152,"data_stored":462232488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510780,"kb_used_data":194476,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93861060,"statfs":{"total":96636764160,"available":96113725440,"internally_reserved":0,"allocated":199143424,"data_stored":198861596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790051,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552896,"kb_used_data":97768,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93818944,"statfs":{"total":96636764160,"available":96070598656,"internally_reserved":0,"allocated":100114432,"data_stored":99792397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590824,"kb_used_data":162132,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93781016,"statfs":{"total":96636764160,"available":96031760384,"internally_reserved":0,"allocated":166023168,"data_stored":165741335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,6,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164438016,"data_stored":164401886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":99221504,"data_stored":99145172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198725632,"data_stored":198685430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:41.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:41.638 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:41.640 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:41.668 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-11T01:23:40.630514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121770761,"num_objects":29741,"num_object_clones":0,"num_object_copies":59482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29741,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":255504,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12656,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1658500,"kb_used_data":440304,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281457020,"statfs":{"total":289910292480,"available":288211988480,"internally_reserved":0,"allocated":450871296,"data_stored":449985600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1247333383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,8,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24223744,"num_objects":-5914,"num_object_clones":0,"num_object_copies":-11828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005270"},"pg_stats":[{"pgid":"2.f","version":"43'18379","reported_seq":18603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.668176+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:38.668176+0000","last_peered":"2024-09-11T01:23:38.668176+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:38.668176+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614464,"num_objects":1859,"num_object_clones":0,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18379,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2189}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18562","reported_seq":18644,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.000002+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:36.000002+0000","last_peered":"2024-09-11T01:23:36.000002+0000","last_clean":"2024-09-11T01:23:36.000002+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:36.000002+0000","last_undegraded":"2024-09-11T01:23:36.000002+0000","last_fullsized":"2024-09-11T01:23:36.000002+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8029126,"num_objects":1962,"num_object_clones":0,"num_object_copies":3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16660,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'18204","reported_seq":18368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.088377+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:36.088377+0000","last_peered":"2024-09-11T01:23:36.088377+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:36.088377+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7540760,"num_objects":1842,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16265,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18419","reported_seq":18466,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.090146+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:36.090146+0000","last_peered":"2024-09-11T01:23:36.090146+0000","last_clean":"2024-09-11T01:23:36.090146+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:36.090146+0000","last_undegraded":"2024-09-11T01:23:36.090146+0000","last_fullsized":"2024-09-11T01:23:36.090146+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7958528,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18419,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18439","reported_seq":18562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:38.654344+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:38.654344+0000","last_peered":"2024-09-11T01:23:38.654344+0000","last_clean":"2024-09-11T01:23:38.654344+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:38.654344+0000","last_undegraded":"2024-09-11T01:23:38.654344+0000","last_fullsized":"2024-09-11T01:23:38.654344+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7528931,"num_objects":1839,"num_object_clones":0,"num_object_copies":3678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15237,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18434","reported_seq":18702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.591748+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:38.591748+0000","last_peered":"2024-09-11T01:23:38.591748+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:38.591748+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7262691,"num_objects":1774,"num_object_clones":0,"num_object_copies":3548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14922,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2275},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18944","reported_seq":19233,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.627136+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:38.627136+0000","last_peered":"2024-09-11T01:23:38.627136+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:38.627136+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7652294,"num_objects":1870,"num_object_clones":0,"num_object_copies":3740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1870,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15697,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2336},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18581","reported_seq":18701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:38.591613+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:38.591613+0000","last_peered":"2024-09-11T01:23:38.591613+0000","last_clean":"2024-09-11T01:23:38.591613+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:38.591613+0000","last_undegraded":"2024-09-11T01:23:38.591613+0000","last_fullsized":"2024-09-11T01:23:38.591613+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18436","reported_seq":26780,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.843284+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:36.843284+0000","last_peered":"2024-09-11T01:23:36.843284+0000","last_clean":"2024-09-11T01:23:36.843284+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:36.843284+0000","last_undegraded":"2024-09-11T01:23:36.843284+0000","last_fullsized":"2024-09-11T01:23:36.843284+0000","mapping_epoch":40,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426531,"num_objects":1814,"num_object_clones":0,"num_object_copies":3628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16303,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18748","reported_seq":19029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.597114+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:38.597114+0000","last_peered":"2024-09-11T01:23:38.597114+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:38.597114+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7561216,"num_objects":1846,"num_object_clones":0,"num_object_copies":3692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15112,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18197","reported_seq":25872,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.165956+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:36.165956+0000","last_peered":"2024-09-11T01:23:36.165956+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:36.165956+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7954915,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15966,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3789,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18683","reported_seq":18837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.179408+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:36.179408+0000","last_peered":"2024-09-11T01:23:36.179408+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:36.179408+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16320,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2268}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18507","reported_seq":26979,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.851893+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:36.851893+0000","last_peered":"2024-09-11T01:23:36.851893+0000","last_clean":"2024-09-11T01:23:36.851893+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:36.851893+0000","last_undegraded":"2024-09-11T01:23:36.851893+0000","last_fullsized":"2024-09-11T01:23:36.851893+0000","mapping_epoch":42,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7557120,"num_objects":1845,"num_object_clones":0,"num_object_copies":3690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15197,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18863","reported_seq":19148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.663907+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:38.663907+0000","last_peered":"2024-09-11T01:23:38.663907+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:38.663907+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"40'8800","ondisk_log_start":"40'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7303168,"num_objects":1783,"num_object_clones":0,"num_object_copies":3566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18688","reported_seq":18989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.640577+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:38.640577+0000","last_peered":"2024-09-11T01:23:38.640577+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:38.640577+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831552,"num_objects":1912,"num_object_clones":0,"num_object_copies":3824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15223,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18355","reported_seq":18596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.602854+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:38.602854+0000","last_peered":"2024-09-11T01:23:38.602854+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:38.602854+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7304134,"num_objects":1785,"num_object_clones":0,"num_object_copies":3570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15012,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2317},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":121770742,"num_objects":29739,"num_object_clones":0,"num_object_copies":59478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29739,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":255502,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12655,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447979520,"data_stored":447826856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510780,"kb_used_data":194476,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93861060,"statfs":{"total":96636764160,"available":96113725440,"internally_reserved":0,"allocated":199143424,"data_stored":198861596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790052,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555580,"kb_used_data":90708,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93816260,"statfs":{"total":96636764160,"available":96067850240,"internally_reserved":0,"allocated":92884992,"data_stored":92562957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592140,"kb_used_data":155120,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93779700,"statfs":{"total":96636764160,"available":96030412800,"internally_reserved":0,"allocated":158842880,"data_stored":158561047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157257728,"data_stored":157221598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":91996160,"data_stored":91919828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198725632,"data_stored":198685430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:41.669 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:42.272 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:42.272 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:42.298 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-11T01:23:40.630514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":121770761,"num_objects":29741,"num_object_clones":0,"num_object_copies":59482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29741,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":255504,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12656,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1658500,"kb_used_data":440304,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281457020,"statfs":{"total":289910292480,"available":288211988480,"internally_reserved":0,"allocated":450871296,"data_stored":449985600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1247333383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,8,14,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24223744,"num_objects":-5914,"num_object_clones":0,"num_object_copies":-11828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005270"},"pg_stats":[{"pgid":"2.f","version":"43'18379","reported_seq":18603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.668176+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:38.668176+0000","last_peered":"2024-09-11T01:23:38.668176+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:38.668176+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614464,"num_objects":1859,"num_object_clones":0,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18379,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2189}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18562","reported_seq":18644,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.000002+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:36.000002+0000","last_peered":"2024-09-11T01:23:36.000002+0000","last_clean":"2024-09-11T01:23:36.000002+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:36.000002+0000","last_undegraded":"2024-09-11T01:23:36.000002+0000","last_fullsized":"2024-09-11T01:23:36.000002+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8029126,"num_objects":1962,"num_object_clones":0,"num_object_copies":3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1962,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16660,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'18204","reported_seq":18368,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.088377+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:36.088377+0000","last_peered":"2024-09-11T01:23:36.088377+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:36.088377+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8200","ondisk_log_start":"39'8200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7540760,"num_objects":1842,"num_object_clones":0,"num_object_copies":3684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16265,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18419","reported_seq":18466,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.090146+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:36.090146+0000","last_peered":"2024-09-11T01:23:36.090146+0000","last_clean":"2024-09-11T01:23:36.090146+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:36.090146+0000","last_undegraded":"2024-09-11T01:23:36.090146+0000","last_fullsized":"2024-09-11T01:23:36.090146+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7958528,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18419,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18439","reported_seq":18562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:38.654344+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:38.654344+0000","last_peered":"2024-09-11T01:23:38.654344+0000","last_clean":"2024-09-11T01:23:38.654344+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:38.654344+0000","last_undegraded":"2024-09-11T01:23:38.654344+0000","last_fullsized":"2024-09-11T01:23:38.654344+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7528931,"num_objects":1839,"num_object_clones":0,"num_object_copies":3678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15237,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18434","reported_seq":18702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.591748+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:38.591748+0000","last_peered":"2024-09-11T01:23:38.591748+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:38.591748+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7262691,"num_objects":1774,"num_object_clones":0,"num_object_copies":3548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14922,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2275},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18944","reported_seq":19233,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.627136+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:38.627136+0000","last_peered":"2024-09-11T01:23:38.627136+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:38.627136+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7652294,"num_objects":1870,"num_object_clones":0,"num_object_copies":3740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1870,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15697,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2336},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18581","reported_seq":18701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:38.591613+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:38.591613+0000","last_peered":"2024-09-11T01:23:38.591613+0000","last_clean":"2024-09-11T01:23:38.591613+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:38.591613+0000","last_undegraded":"2024-09-11T01:23:38.591613+0000","last_fullsized":"2024-09-11T01:23:38.591613+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18436","reported_seq":26780,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.843284+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:36.843284+0000","last_peered":"2024-09-11T01:23:36.843284+0000","last_clean":"2024-09-11T01:23:36.843284+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:36.843284+0000","last_undegraded":"2024-09-11T01:23:36.843284+0000","last_fullsized":"2024-09-11T01:23:36.843284+0000","mapping_epoch":40,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426531,"num_objects":1814,"num_object_clones":0,"num_object_copies":3628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16303,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18748","reported_seq":19029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.597114+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:38.597114+0000","last_peered":"2024-09-11T01:23:38.597114+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:38.597114+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7561216,"num_objects":1846,"num_object_clones":0,"num_object_copies":3692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15112,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18197","reported_seq":25872,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.165956+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:36.165956+0000","last_peered":"2024-09-11T01:23:36.165956+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:36.165956+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'8100","ondisk_log_start":"39'8100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7954915,"num_objects":1943,"num_object_clones":0,"num_object_copies":3886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15966,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3789,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1704}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680196+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:20:45.680196+0000","last_peered":"2024-09-11T01:20:45.680196+0000","last_clean":"2024-09-11T01:20:45.680196+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:20:45.680196+0000","last_undegraded":"2024-09-11T01:20:45.680196+0000","last_fullsized":"2024-09-11T01:20:45.680196+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18683","reported_seq":18837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:36.179408+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:36.179408+0000","last_peered":"2024-09-11T01:23:36.179408+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:36.179408+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7696384,"num_objects":1879,"num_object_clones":0,"num_object_copies":3758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1879,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16320,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2268}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18507","reported_seq":26979,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:36.851893+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:36.851893+0000","last_peered":"2024-09-11T01:23:36.851893+0000","last_clean":"2024-09-11T01:23:36.851893+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:36.851893+0000","last_undegraded":"2024-09-11T01:23:36.851893+0000","last_fullsized":"2024-09-11T01:23:36.851893+0000","mapping_epoch":42,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7557120,"num_objects":1845,"num_object_clones":0,"num_object_copies":3690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15197,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680283+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:20:45.680283+0000","last_peered":"2024-09-11T01:20:45.680283+0000","last_clean":"2024-09-11T01:20:45.680283+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:20:45.680283+0000","last_undegraded":"2024-09-11T01:20:45.680283+0000","last_fullsized":"2024-09-11T01:20:45.680283+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18863","reported_seq":19148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.663907+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:38.663907+0000","last_peered":"2024-09-11T01:23:38.663907+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:38.663907+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"40'8800","ondisk_log_start":"40'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7303168,"num_objects":1783,"num_object_clones":0,"num_object_copies":3566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18688","reported_seq":18989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.640577+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:38.640577+0000","last_peered":"2024-09-11T01:23:38.640577+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:38.640577+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831552,"num_objects":1912,"num_object_clones":0,"num_object_copies":3824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15223,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680392+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:20:45.680392+0000","last_peered":"2024-09-11T01:20:45.680392+0000","last_clean":"2024-09-11T01:20:45.680392+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:20:45.680392+0000","last_undegraded":"2024-09-11T01:20:45.680392+0000","last_fullsized":"2024-09-11T01:20:45.680392+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18355","reported_seq":18596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.602854+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:38.602854+0000","last_peered":"2024-09-11T01:23:38.602854+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:38.602854+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7304134,"num_objects":1785,"num_object_clones":0,"num_object_copies":3570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15012,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2317},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":121770742,"num_objects":29739,"num_object_clones":0,"num_object_copies":59478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29739,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":255502,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12655,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447979520,"data_stored":447826856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510780,"kb_used_data":194476,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93861060,"statfs":{"total":96636764160,"available":96113725440,"internally_reserved":0,"allocated":199143424,"data_stored":198861596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790052,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555580,"kb_used_data":90708,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93816260,"statfs":{"total":96636764160,"available":96067850240,"internally_reserved":0,"allocated":92884992,"data_stored":92562957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592140,"kb_used_data":155120,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93779700,"statfs":{"total":96636764160,"available":96030412800,"internally_reserved":0,"allocated":158842880,"data_stored":158561047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157257728,"data_stored":157221598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":91996160,"data_stored":91919828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":198725632,"data_stored":198685430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:42.299 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:42.299 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'18379", 'reported_seq': 18603, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.668176+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:38.668176+0000', 'last_peered': '2024-09-11T01:23:38.668176+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:38.668176+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "39'8300", 'ondisk_log_start': "39'8300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7614464, 'num_objects': 1859, 'num_object_clones': 0, 'num_object_copies': 3718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18379, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2189}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'18204", 'reported_seq': 18368, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:36.088377+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:36.088377+0000', 'last_peered': '2024-09-11T01:23:36.088377+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:36.088377+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'8200", 'ondisk_log_start': "39'8200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7540760, 'num_objects': 1842, 'num_object_clones': 0, 'num_object_copies': 3684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1842, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16265, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'18434", 'reported_seq': 18702, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.591748+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:38.591748+0000', 'last_peered': '2024-09-11T01:23:38.591748+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:38.591748+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'8400", 'ondisk_log_start': "39'8400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7262691, 'num_objects': 1774, 'num_object_clones': 0, 'num_object_copies': 3548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1774, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14922, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2275}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'18944", 'reported_seq': 19233, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.627136+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:38.627136+0000', 'last_peered': '2024-09-11T01:23:38.627136+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:38.627136+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'8900", 'ondisk_log_start': "41'8900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7652294, 'num_objects': 1870, 'num_object_clones': 0, 'num_object_copies': 3740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1870, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15697, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2336}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'18748", 'reported_seq': 19029, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.597114+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:38.597114+0000', 'last_peered': '2024-09-11T01:23:38.597114+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:38.597114+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'8700", 'ondisk_log_start': "41'8700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7561216, 'num_objects': 1846, 'num_object_clones': 0, 'num_object_copies': 3692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1846, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15112, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2184}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:42.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'18197", 'reported_seq': 25872, 'reported_epoch': 43, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:36.165956+0000', 'last_change': '2024-09-11T01:20:43.944862+0000', 'last_active': '2024-09-11T01:23:36.165956+0000', 'last_peered': '2024-09-11T01:23:36.165956+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:36.165956+0000', 'last_undegraded': '2024-09-11T01:15:20.032380+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 38, 'log_start': "39'8100", 'ondisk_log_start': "39'8100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7954915, 'num_objects': 1943, 'num_object_clones': 0, 'num_object_copies': 3886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 239, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1943, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15966, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3789, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1704}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'18683", 'reported_seq': 18837, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:36.179408+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:36.179408+0000', 'last_peered': '2024-09-11T01:23:36.179408+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:36.179408+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "39'8600", 'ondisk_log_start': "39'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7696384, 'num_objects': 1879, 'num_object_clones': 0, 'num_object_copies': 3758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1879, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16320, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2268}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'18863", 'reported_seq': 19148, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.663907+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:38.663907+0000', 'last_peered': '2024-09-11T01:23:38.663907+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:38.663907+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "40'8800", 'ondisk_log_start': "40'8800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7303168, 'num_objects': 1783, 'num_object_clones': 0, 'num_object_copies': 3566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1783, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15256, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2334}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'18688", 'reported_seq': 18989, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.640577+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:38.640577+0000', 'last_peered': '2024-09-11T01:23:38.640577+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:38.640577+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "39'8600", 'ondisk_log_start': "39'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7831552, 'num_objects': 1912, 'num_object_clones': 0, 'num_object_copies': 3824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1912, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15223, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2153}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:42.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'18355", 'reported_seq': 18596, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.602854+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:38.602854+0000', 'last_peered': '2024-09-11T01:23:38.602854+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:38.602854+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'8300", 'ondisk_log_start': "39'8300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7304134, 'num_objects': 1785, 'num_object_clones': 0, 'num_object_copies': 3570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1785, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15012, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2317}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:42.302 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:43.018 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:43.018 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36850/57542 objects degraded (64.040%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":623,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28771,"data_bytes":117797641,"bytes_used":1694941184,"bytes_avail":288215351296,"bytes_total":289910292480,"degraded_objects":36850,"degraded_total":57542,"degraded_ratio":0.64040179347259396,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":405,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":464},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:43.048 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:43.048 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:43.605 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:43.607 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:43.630 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-09-11T01:23:42.631324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117797641,"num_objects":28771,"num_object_clones":0,"num_object_copies":57542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28771,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":256474,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12785,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1655216,"kb_used_data":428724,"kb_used_omap":12,"kb_used_meta":1226419,"kb_avail":281460304,"statfs":{"total":289910292480,"available":288215351296,"internally_reserved":0,"allocated":439013376,"data_stored":438127680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1255853063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,7,8,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22831104,"num_objects":-5574,"num_object_clones":0,"num_object_copies":-11148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5574,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":405,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005194"},"pg_stats":[{"pgid":"2.f","version":"43'18379","reported_seq":18603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.668176+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:38.668176+0000","last_peered":"2024-09-11T01:23:38.668176+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:38.668176+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7614464,"num_objects":1859,"num_object_clones":0,"num_object_copies":3718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18379,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2189}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'18706","reported_seq":18788,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.141220+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:41.141220+0000","last_peered":"2024-09-11T01:23:41.141220+0000","last_clean":"2024-09-11T01:23:41.141220+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:41.141220+0000","last_undegraded":"2024-09-11T01:23:41.141220+0000","last_fullsized":"2024-09-11T01:23:41.141220+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7439302,"num_objects":1818,"num_object_clones":0,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16804,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'18332","reported_seq":18501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:41.054475+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:41.054475+0000","last_peered":"2024-09-11T01:23:41.054475+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:41.054475+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7016472,"num_objects":1714,"num_object_clones":0,"num_object_copies":3428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16393,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2249}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'18552","reported_seq":18599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.044599+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:41.044599+0000","last_peered":"2024-09-11T01:23:41.044599+0000","last_clean":"2024-09-11T01:23:41.044599+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:41.044599+0000","last_undegraded":"2024-09-11T01:23:41.044599+0000","last_fullsized":"2024-09-11T01:23:41.044599+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7413760,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18552,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18439","reported_seq":18562,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:38.654344+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:38.654344+0000","last_peered":"2024-09-11T01:23:38.654344+0000","last_clean":"2024-09-11T01:23:38.654344+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:38.654344+0000","last_undegraded":"2024-09-11T01:23:38.654344+0000","last_fullsized":"2024-09-11T01:23:38.654344+0000","mapping_epoch":31,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7528931,"num_objects":1839,"num_object_clones":0,"num_object_copies":3678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1839,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15237,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18434","reported_seq":18702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.591748+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:38.591748+0000","last_peered":"2024-09-11T01:23:38.591748+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:38.591748+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7262691,"num_objects":1774,"num_object_clones":0,"num_object_copies":3548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14922,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2275},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'18944","reported_seq":19233,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.627136+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:38.627136+0000","last_peered":"2024-09-11T01:23:38.627136+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:38.627136+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7652294,"num_objects":1870,"num_object_clones":0,"num_object_copies":3740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1870,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15697,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2336},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18581","reported_seq":18701,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:38.591613+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:38.591613+0000","last_peered":"2024-09-11T01:23:38.591613+0000","last_clean":"2024-09-11T01:23:38.591613+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:38.591613+0000","last_undegraded":"2024-09-11T01:23:38.591613+0000","last_fullsized":"2024-09-11T01:23:38.591613+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15534,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18563","reported_seq":26907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.901844+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:41.901844+0000","last_peered":"2024-09-11T01:23:41.901844+0000","last_clean":"2024-09-11T01:23:41.901844+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:41.901844+0000","last_undegraded":"2024-09-11T01:23:41.901844+0000","last_fullsized":"2024-09-11T01:23:41.901844+0000","mapping_epoch":40,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6906339,"num_objects":1687,"num_object_clones":0,"num_object_copies":3374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16430,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677042+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:20:45.677042+0000","last_peered":"2024-09-11T01:20:45.677042+0000","last_clean":"2024-09-11T01:20:45.677042+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:20:45.677042+0000","last_undegraded":"2024-09-11T01:20:45.677042+0000","last_fullsized":"2024-09-11T01:20:45.677042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18748","reported_seq":19029,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.597114+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:38.597114+0000","last_peered":"2024-09-11T01:23:38.597114+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:38.597114+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7561216,"num_objects":1846,"num_object_clones":0,"num_object_copies":3692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15112,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2184}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18336","reported_seq":26269,"reported_epoch":43,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:41.174708+0000","last_change":"2024-09-11T01:20:43.944862+0000","last_active":"2024-09-11T01:23:41.174708+0000","last_peered":"2024-09-11T01:23:41.174708+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:41.174708+0000","last_undegraded":"2024-09-11T01:15:20.032380+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7385571,"num_objects":1804,"num_object_clones":0,"num_object_copies":3608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":110,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1804,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16105,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3918,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1694}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466587+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:21:35.466587+0000","last_peered":"2024-09-11T01:21:35.466587+0000","last_clean":"2024-09-11T01:21:35.466587+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:21:35.466587+0000","last_undegraded":"2024-09-11T01:21:35.466587+0000","last_fullsized":"2024-09-11T01:21:35.466587+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'18828","reported_seq":18982,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:41.105965+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:41.105965+0000","last_peered":"2024-09-11T01:23:41.105965+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:41.105965+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7102464,"num_objects":1734,"num_object_clones":0,"num_object_copies":3468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16465,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2413}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18661","reported_seq":27133,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.937664+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:41.937664+0000","last_peered":"2024-09-11T01:23:41.937664+0000","last_clean":"2024-09-11T01:23:41.937664+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:41.937664+0000","last_undegraded":"2024-09-11T01:23:41.937664+0000","last_fullsized":"2024-09-11T01:23:41.937664+0000","mapping_epoch":42,"log_start":"40'8600","ondisk_log_start":"40'8600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466612+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:21:35.466612+0000","last_peered":"2024-09-11T01:21:35.466612+0000","last_clean":"2024-09-11T01:21:35.466612+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:21:35.466612+0000","last_undegraded":"2024-09-11T01:21:35.466612+0000","last_fullsized":"2024-09-11T01:21:35.466612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'18863","reported_seq":19148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.663907+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:38.663907+0000","last_peered":"2024-09-11T01:23:38.663907+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:38.663907+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"40'8800","ondisk_log_start":"40'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7303168,"num_objects":1783,"num_object_clones":0,"num_object_copies":3566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15256,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2334}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.677374+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:20:45.677374+0000","last_peered":"2024-09-11T01:20:45.677374+0000","last_clean":"2024-09-11T01:20:45.677374+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:20:45.677374+0000","last_undegraded":"2024-09-11T01:20:45.677374+0000","last_fullsized":"2024-09-11T01:20:45.677374+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18688","reported_seq":18989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.640577+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:38.640577+0000","last_peered":"2024-09-11T01:23:38.640577+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:38.640577+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"39'8600","ondisk_log_start":"39'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7831552,"num_objects":1912,"num_object_clones":0,"num_object_copies":3824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15223,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466625+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:21:35.466625+0000","last_peered":"2024-09-11T01:21:35.466625+0000","last_clean":"2024-09-11T01:21:35.466625+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:21:35.466625+0000","last_undegraded":"2024-09-11T01:21:35.466625+0000","last_fullsized":"2024-09-11T01:21:35.466625+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18355","reported_seq":18596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.602854+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:38.602854+0000","last_peered":"2024-09-11T01:23:38.602854+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:38.602854+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8300","ondisk_log_start":"39'8300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7304134,"num_objects":1785,"num_object_clones":0,"num_object_copies":3570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15012,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2317},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":117797622,"num_objects":28769,"num_object_clones":0,"num_object_copies":57538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36850,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28769,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":256472,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12784,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":436219904,"data_stored":436067240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160809,"ondisk_log_size":160809,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510520,"kb_used_data":190056,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93861320,"statfs":{"total":96636764160,"available":96113991680,"internally_reserved":0,"allocated":194617344,"data_stored":194335516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790053,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552556,"kb_used_data":83548,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93819284,"statfs":{"total":96636764160,"available":96070946816,"internally_reserved":0,"allocated":85553152,"data_stored":85231117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592140,"kb_used_data":155120,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93779700,"statfs":{"total":96636764160,"available":96030412800,"internally_reserved":0,"allocated":158842880,"data_stored":158561047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,5,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157257728,"data_stored":157221598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":84738048,"data_stored":84661716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":194224128,"data_stored":194183926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:46.633 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:47.003 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:47.003 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:47.028 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-11T01:23:46.838375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":109421321,"num_objects":26726,"num_object_clones":0,"num_object_copies":53452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26726,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":258519,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12895,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1635284,"kb_used_data":417096,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281480236,"statfs":{"total":289910292480,"available":288235761664,"internally_reserved":0,"allocated":427106304,"data_stored":426215368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1247333383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,1,5,7,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17752064,"num_objects":-4334,"num_object_clones":0,"num_object_copies":-8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4334,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.208779"},"pg_stats":[{"pgid":"2.f","version":"43'18521","reported_seq":18746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.640342+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:43.640342+0000","last_peered":"2024-09-11T01:23:43.640342+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:43.640342+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"40'8500","ondisk_log_start":"40'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7032832,"num_objects":1717,"num_object_clones":0,"num_object_copies":3434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18521,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2331}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'18858","reported_seq":18942,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.054414+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:46.054414+0000","last_peered":"2024-09-11T01:23:46.054414+0000","last_clean":"2024-09-11T01:23:46.054414+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:46.054414+0000","last_undegraded":"2024-09-11T01:23:46.054414+0000","last_fullsized":"2024-09-11T01:23:46.054414+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6816710,"num_objects":1666,"num_object_clones":0,"num_object_copies":3332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16956,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'18461","reported_seq":18632,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:46.106774+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:46.106774+0000","last_peered":"2024-09-11T01:23:46.106774+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:46.106774+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488088,"num_objects":1585,"num_object_clones":0,"num_object_copies":3170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2378}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'18723","reported_seq":18772,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.169582+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:46.169582+0000","last_peered":"2024-09-11T01:23:46.169582+0000","last_clean":"2024-09-11T01:23:46.169582+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:46.169582+0000","last_undegraded":"2024-09-11T01:23:46.169582+0000","last_fullsized":"2024-09-11T01:23:46.169582+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6713344,"num_objects":1639,"num_object_clones":0,"num_object_copies":3278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18723,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18574","reported_seq":18697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:43.591420+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:43.591420+0000","last_peered":"2024-09-11T01:23:43.591420+0000","last_clean":"2024-09-11T01:23:43.591420+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:43.591420+0000","last_undegraded":"2024-09-11T01:23:43.591420+0000","last_fullsized":"2024-09-11T01:23:43.591420+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6975971,"num_objects":1704,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15372,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18585","reported_seq":18853,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.645523+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:43.645523+0000","last_peered":"2024-09-11T01:23:43.645523+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:43.645523+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6644195,"num_objects":1623,"num_object_clones":0,"num_object_copies":3246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15073,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2426},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'19101","reported_seq":19384,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.683250+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:43.683250+0000","last_peered":"2024-09-11T01:23:43.683250+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:43.683250+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7009222,"num_objects":1713,"num_object_clones":0,"num_object_copies":3426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1713,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15854,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2493},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18724","reported_seq":18845,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:43.622819+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:43.622819+0000","last_peered":"2024-09-11T01:23:43.622819+0000","last_clean":"2024-09-11T01:23:43.622819+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:43.622819+0000","last_undegraded":"2024-09-11T01:23:43.622819+0000","last_fullsized":"2024-09-11T01:23:43.622819+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18563","reported_seq":26907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.901844+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:41.901844+0000","last_peered":"2024-09-11T01:23:41.901844+0000","last_clean":"2024-09-11T01:23:41.901844+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:41.901844+0000","last_undegraded":"2024-09-11T01:23:41.901844+0000","last_fullsized":"2024-09-11T01:23:41.901844+0000","mapping_epoch":40,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6906339,"num_objects":1687,"num_object_clones":0,"num_object_copies":3374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16430,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.840123+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:45.840123+0000","last_peered":"2024-09-11T01:23:45.840123+0000","last_clean":"2024-09-11T01:23:45.840123+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:45.840123+0000","last_undegraded":"2024-09-11T01:23:45.840123+0000","last_fullsized":"2024-09-11T01:23:45.840123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18899","reported_seq":19180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.645556+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:43.645556+0000","last_peered":"2024-09-11T01:23:43.645556+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:43.645556+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18461","reported_seq":26615,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-11T01:23:45.838894+0000","last_change":"2024-09-11T01:23:45.850699+0000","last_active":"2024-09-11T01:23:45.850699+0000","last_peered":"2024-09-11T01:23:45.838894+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:45.838894+0000","last_undegraded":"2024-09-11T01:23:45.838894+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":44,"log_start":"40'8400","ondisk_log_start":"40'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873571,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16230,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466587+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:21:35.466587+0000","last_peered":"2024-09-11T01:21:35.466587+0000","last_clean":"2024-09-11T01:21:35.466587+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:21:35.466587+0000","last_undegraded":"2024-09-11T01:21:35.466587+0000","last_fullsized":"2024-09-11T01:21:35.466587+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'18972","reported_seq":19129,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:46.169429+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:46.169429+0000","last_peered":"2024-09-11T01:23:46.169429+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:46.169429+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":1590,"num_object_clones":0,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16609,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18661","reported_seq":27133,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.937664+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:41.937664+0000","last_peered":"2024-09-11T01:23:41.937664+0000","last_clean":"2024-09-11T01:23:41.937664+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:41.937664+0000","last_undegraded":"2024-09-11T01:23:41.937664+0000","last_fullsized":"2024-09-11T01:23:41.937664+0000","mapping_epoch":42,"log_start":"40'8600","ondisk_log_start":"40'8600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466612+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:21:35.466612+0000","last_peered":"2024-09-11T01:21:35.466612+0000","last_clean":"2024-09-11T01:21:35.466612+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:21:35.466612+0000","last_undegraded":"2024-09-11T01:21:35.466612+0000","last_fullsized":"2024-09-11T01:21:35.466612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'19014","reported_seq":19300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.591074+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:43.591074+0000","last_peered":"2024-09-11T01:23:43.591074+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:43.591074+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":1632,"num_object_clones":0,"num_object_copies":3264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15407,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.843386+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:45.843386+0000","last_peered":"2024-09-11T01:23:45.843386+0000","last_clean":"2024-09-11T01:23:45.843386+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:45.843386+0000","last_undegraded":"2024-09-11T01:23:45.843386+0000","last_fullsized":"2024-09-11T01:23:45.843386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18847","reported_seq":19146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.639555+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:43.639555+0000","last_peered":"2024-09-11T01:23:43.639555+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:43.639555+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7180288,"num_objects":1753,"num_object_clones":0,"num_object_copies":3506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15382,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466625+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:21:35.466625+0000","last_peered":"2024-09-11T01:21:35.466625+0000","last_clean":"2024-09-11T01:21:35.466625+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:21:35.466625+0000","last_undegraded":"2024-09-11T01:21:35.466625+0000","last_fullsized":"2024-09-11T01:21:35.466625+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18490","reported_seq":18731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.664106+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:43.664106+0000","last_peered":"2024-09-11T01:23:43.664106+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:43.664106+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6751174,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15147,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":109421302,"num_objects":26724,"num_object_clones":0,"num_object_copies":53448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26724,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":258517,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12894,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":424316928,"data_stored":424164264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510092,"kb_used_data":185464,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93861748,"statfs":{"total":96636764160,"available":96114429952,"internally_reserved":0,"allocated":189915136,"data_stored":189628068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790053,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552556,"kb_used_data":83548,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93819284,"statfs":{"total":96636764160,"available":96070946816,"internally_reserved":0,"allocated":85553152,"data_stored":85231117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572636,"kb_used_data":148084,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93799204,"statfs":{"total":96636764160,"available":96050384896,"internally_reserved":0,"allocated":151638016,"data_stored":151356183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150061056,"data_stored":150024926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":84738048,"data_stored":84661716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189517824,"data_stored":189477622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:47.029 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:47.628 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:47.630 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:47.661 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-11T01:23:46.838375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":109421321,"num_objects":26726,"num_object_clones":0,"num_object_copies":53452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26726,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":258519,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12895,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1635284,"kb_used_data":417096,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281480236,"statfs":{"total":289910292480,"available":288235761664,"internally_reserved":0,"allocated":427106304,"data_stored":426215368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1247333383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,3,1,5,7,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17752064,"num_objects":-4334,"num_object_clones":0,"num_object_copies":-8668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4334,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.208779"},"pg_stats":[{"pgid":"2.f","version":"43'18521","reported_seq":18746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.640342+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:43.640342+0000","last_peered":"2024-09-11T01:23:43.640342+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:43.640342+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"40'8500","ondisk_log_start":"40'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7032832,"num_objects":1717,"num_object_clones":0,"num_object_copies":3434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18521,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2331}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'18858","reported_seq":18942,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.054414+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:46.054414+0000","last_peered":"2024-09-11T01:23:46.054414+0000","last_clean":"2024-09-11T01:23:46.054414+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:46.054414+0000","last_undegraded":"2024-09-11T01:23:46.054414+0000","last_fullsized":"2024-09-11T01:23:46.054414+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6816710,"num_objects":1666,"num_object_clones":0,"num_object_copies":3332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16956,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'18461","reported_seq":18632,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:46.106774+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:46.106774+0000","last_peered":"2024-09-11T01:23:46.106774+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:46.106774+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488088,"num_objects":1585,"num_object_clones":0,"num_object_copies":3170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2378}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'18723","reported_seq":18772,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.169582+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:46.169582+0000","last_peered":"2024-09-11T01:23:46.169582+0000","last_clean":"2024-09-11T01:23:46.169582+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:46.169582+0000","last_undegraded":"2024-09-11T01:23:46.169582+0000","last_fullsized":"2024-09-11T01:23:46.169582+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6713344,"num_objects":1639,"num_object_clones":0,"num_object_copies":3278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18723,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18574","reported_seq":18697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:43.591420+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:43.591420+0000","last_peered":"2024-09-11T01:23:43.591420+0000","last_clean":"2024-09-11T01:23:43.591420+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:43.591420+0000","last_undegraded":"2024-09-11T01:23:43.591420+0000","last_fullsized":"2024-09-11T01:23:43.591420+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6975971,"num_objects":1704,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15372,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18585","reported_seq":18853,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.645523+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:43.645523+0000","last_peered":"2024-09-11T01:23:43.645523+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:43.645523+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6644195,"num_objects":1623,"num_object_clones":0,"num_object_copies":3246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15073,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2426},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'19101","reported_seq":19384,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.683250+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:43.683250+0000","last_peered":"2024-09-11T01:23:43.683250+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:43.683250+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7009222,"num_objects":1713,"num_object_clones":0,"num_object_copies":3426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1713,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15854,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2493},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18724","reported_seq":18845,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:43.622819+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:43.622819+0000","last_peered":"2024-09-11T01:23:43.622819+0000","last_clean":"2024-09-11T01:23:43.622819+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:43.622819+0000","last_undegraded":"2024-09-11T01:23:43.622819+0000","last_fullsized":"2024-09-11T01:23:43.622819+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.356969+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:21:20.356969+0000","last_peered":"2024-09-11T01:21:20.356969+0000","last_clean":"2024-09-11T01:21:20.356969+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:21:20.356969+0000","last_undegraded":"2024-09-11T01:21:20.356969+0000","last_fullsized":"2024-09-11T01:21:20.356969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'18563","reported_seq":26907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.901844+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:41.901844+0000","last_peered":"2024-09-11T01:23:41.901844+0000","last_clean":"2024-09-11T01:23:41.901844+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:41.901844+0000","last_undegraded":"2024-09-11T01:23:41.901844+0000","last_fullsized":"2024-09-11T01:23:41.901844+0000","mapping_epoch":40,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6906339,"num_objects":1687,"num_object_clones":0,"num_object_copies":3374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1687,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16430,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.840123+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:45.840123+0000","last_peered":"2024-09-11T01:23:45.840123+0000","last_clean":"2024-09-11T01:23:45.840123+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:45.840123+0000","last_undegraded":"2024-09-11T01:23:45.840123+0000","last_fullsized":"2024-09-11T01:23:45.840123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18899","reported_seq":19180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.645556+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:43.645556+0000","last_peered":"2024-09-11T01:23:43.645556+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:43.645556+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18461","reported_seq":26615,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-11T01:23:45.838894+0000","last_change":"2024-09-11T01:23:45.850699+0000","last_active":"2024-09-11T01:23:45.850699+0000","last_peered":"2024-09-11T01:23:45.838894+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:45.838894+0000","last_undegraded":"2024-09-11T01:23:45.838894+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":44,"log_start":"40'8400","ondisk_log_start":"40'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873571,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16230,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466587+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:21:35.466587+0000","last_peered":"2024-09-11T01:21:35.466587+0000","last_clean":"2024-09-11T01:21:35.466587+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:21:35.466587+0000","last_undegraded":"2024-09-11T01:21:35.466587+0000","last_fullsized":"2024-09-11T01:21:35.466587+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'18972","reported_seq":19129,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:46.169429+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:46.169429+0000","last_peered":"2024-09-11T01:23:46.169429+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:46.169429+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":1590,"num_object_clones":0,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16609,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'18661","reported_seq":27133,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:41.937664+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:41.937664+0000","last_peered":"2024-09-11T01:23:41.937664+0000","last_clean":"2024-09-11T01:23:41.937664+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:41.937664+0000","last_undegraded":"2024-09-11T01:23:41.937664+0000","last_fullsized":"2024-09-11T01:23:41.937664+0000","mapping_epoch":42,"log_start":"40'8600","ondisk_log_start":"40'8600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15351,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466612+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:21:35.466612+0000","last_peered":"2024-09-11T01:21:35.466612+0000","last_clean":"2024-09-11T01:21:35.466612+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:21:35.466612+0000","last_undegraded":"2024-09-11T01:21:35.466612+0000","last_fullsized":"2024-09-11T01:21:35.466612+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'19014","reported_seq":19300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.591074+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:43.591074+0000","last_peered":"2024-09-11T01:23:43.591074+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:43.591074+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":1632,"num_object_clones":0,"num_object_copies":3264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15407,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.843386+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:45.843386+0000","last_peered":"2024-09-11T01:23:45.843386+0000","last_clean":"2024-09-11T01:23:45.843386+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:45.843386+0000","last_undegraded":"2024-09-11T01:23:45.843386+0000","last_fullsized":"2024-09-11T01:23:45.843386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18847","reported_seq":19146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.639555+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:43.639555+0000","last_peered":"2024-09-11T01:23:43.639555+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:43.639555+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7180288,"num_objects":1753,"num_object_clones":0,"num_object_copies":3506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15382,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.466625+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:21:35.466625+0000","last_peered":"2024-09-11T01:21:35.466625+0000","last_clean":"2024-09-11T01:21:35.466625+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:21:35.466625+0000","last_undegraded":"2024-09-11T01:21:35.466625+0000","last_fullsized":"2024-09-11T01:21:35.466625+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.680466+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:20:45.680466+0000","last_peered":"2024-09-11T01:20:45.680466+0000","last_clean":"2024-09-11T01:20:45.680466+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:20:45.680466+0000","last_undegraded":"2024-09-11T01:20:45.680466+0000","last_fullsized":"2024-09-11T01:20:45.680466+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18490","reported_seq":18731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.664106+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:43.664106+0000","last_peered":"2024-09-11T01:23:43.664106+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:43.664106+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6751174,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15147,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":109421302,"num_objects":26724,"num_object_clones":0,"num_object_copies":53448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26724,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":258517,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12894,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":424316928,"data_stored":424164264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510092,"kb_used_data":185464,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93861748,"statfs":{"total":96636764160,"available":96114429952,"internally_reserved":0,"allocated":189915136,"data_stored":189628068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790053,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552556,"kb_used_data":83548,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93819284,"statfs":{"total":96636764160,"available":96070946816,"internally_reserved":0,"allocated":85553152,"data_stored":85231117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572636,"kb_used_data":148084,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93799204,"statfs":{"total":96636764160,"available":96050384896,"internally_reserved":0,"allocated":151638016,"data_stored":151356183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150061056,"data_stored":150024926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":84738048,"data_stored":84661716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189517824,"data_stored":189477622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:47.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:47.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'18521", 'reported_seq': 18746, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.640342+0000', 'last_change': '2024-09-11T01:15:20.492716+0000', 'last_active': '2024-09-11T01:23:43.640342+0000', 'last_peered': '2024-09-11T01:23:43.640342+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:43.640342+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "40'8500", 'ondisk_log_start': "40'8500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7032832, 'num_objects': 1717, 'num_object_clones': 0, 'num_object_copies': 3434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18521, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2331}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "44'18461", 'reported_seq': 18632, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:46.106774+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:46.106774+0000', 'last_peered': '2024-09-11T01:23:46.106774+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:46.106774+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "39'8400", 'ondisk_log_start': "39'8400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6488088, 'num_objects': 1585, 'num_object_clones': 0, 'num_object_copies': 3170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1585, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16522, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2378}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'18585", 'reported_seq': 18853, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.645523+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:43.645523+0000', 'last_peered': '2024-09-11T01:23:43.645523+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:43.645523+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "39'8500", 'ondisk_log_start': "39'8500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6644195, 'num_objects': 1623, 'num_object_clones': 0, 'num_object_copies': 3246, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1623, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15073, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2426}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'19101", 'reported_seq': 19384, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.683250+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:43.683250+0000', 'last_peered': '2024-09-11T01:23:43.683250+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:43.683250+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7009222, 'num_objects': 1713, 'num_object_clones': 0, 'num_object_copies': 3426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1713, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15854, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2493}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:47.663 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'18899", 'reported_seq': 19180, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.645556+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:43.645556+0000', 'last_peered': '2024-09-11T01:23:43.645556+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:43.645556+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'8800", 'ondisk_log_start': "41'8800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6942720, 'num_objects': 1695, 'num_object_clones': 0, 'num_object_copies': 3390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15263, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2335}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'18461", 'reported_seq': 26615, 'reported_epoch': 43, 'state': 'peering', 'last_fresh': '2024-09-11T01:23:45.838894+0000', 'last_change': '2024-09-11T01:23:45.850699+0000', 'last_active': '2024-09-11T01:23:45.850699+0000', 'last_peered': '2024-09-11T01:23:45.838894+0000', 'last_clean': '2024-09-11T01:15:17.951087+0000', 'last_became_active': '2024-09-11T01:15:20.508900+0000', 'last_became_peered': '2024-09-11T01:15:20.508900+0000', 'last_unstale': '2024-09-11T01:23:45.838894+0000', 'last_undegraded': '2024-09-11T01:23:45.838894+0000', 'last_fullsized': '2024-09-11T01:15:19.986999+0000', 'mapping_epoch': 44, 'log_start': "40'8400", 'ondisk_log_start': "40'8400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2276", 'last_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:53.159379+0000', 'objects_scrubbed': 2227, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 885, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6873571, 'num_objects': 1679, 'num_object_clones': 0, 'num_object_copies': 3358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1679, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16230, 'num_write_kb': 31364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4028, 'num_bytes_recovered': 2957795, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "44'18972", 'reported_seq': 19129, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:46.169429+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:46.169429+0000', 'last_peered': '2024-09-11T01:23:46.169429+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:46.169429+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'8900", 'ondisk_log_start': "41'8900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6512640, 'num_objects': 1590, 'num_object_clones': 0, 'num_object_copies': 3180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16609, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2557}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'19014", 'reported_seq': 19300, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.591074+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:43.591074+0000', 'last_peered': '2024-09-11T01:23:43.591074+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:43.591074+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9000", 'ondisk_log_start': "41'9000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6684672, 'num_objects': 1632, 'num_object_clones': 0, 'num_object_copies': 3264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15407, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2485}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:47.664 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'18847", 'reported_seq': 19146, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.639555+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:43.639555+0000', 'last_peered': '2024-09-11T01:23:43.639555+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:43.639555+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'8800", 'ondisk_log_start': "41'8800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7180288, 'num_objects': 1753, 'num_object_clones': 0, 'num_object_copies': 3506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15382, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.665 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:47.665 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'18490", 'reported_seq': 18731, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:43.664106+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:43.664106+0000', 'last_peered': '2024-09-11T01:23:43.664106+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:43.664106+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "39'8400", 'ondisk_log_start': "39'8400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6751174, 'num_objects': 1650, 'num_object_clones': 0, 'num_object_copies': 3300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1650, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15147, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2452}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:47.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:48.391 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:48.392 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36740/53452 objects degraded (68.735%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":628,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26726,"data_bytes":109421321,"bytes_used":1674530816,"bytes_avail":288235761664,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":36740,"degraded_total":53452,"degraded_ratio":0.68734565591558772,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":465},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:48.418 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:48.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:49.022 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:49.025 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:49.047 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-11T01:23:48.632933+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":108413705,"num_objects":26480,"num_object_clones":0,"num_object_copies":52960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26480,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":258765,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12895,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1620484,"kb_used_data":410612,"kb_used_omap":12,"kb_used_meta":1209779,"kb_avail":281495036,"statfs":{"total":289910292480,"available":288250916864,"internally_reserved":0,"allocated":420466688,"data_stored":419570512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1238813703},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,0,6,7,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13357056,"num_objects":-3261,"num_object_clones":0,"num_object_copies":-6522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3261,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002418"},"pg_stats":[{"pgid":"2.f","version":"43'18521","reported_seq":18746,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.640342+0000","last_change":"2024-09-11T01:15:20.492716+0000","last_active":"2024-09-11T01:23:43.640342+0000","last_peered":"2024-09-11T01:23:43.640342+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:43.640342+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"40'8500","ondisk_log_start":"40'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7032832,"num_objects":1717,"num_object_clones":0,"num_object_copies":3434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18521,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2331}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'18858","reported_seq":18942,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.054414+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:46.054414+0000","last_peered":"2024-09-11T01:23:46.054414+0000","last_clean":"2024-09-11T01:23:46.054414+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:46.054414+0000","last_undegraded":"2024-09-11T01:23:46.054414+0000","last_fullsized":"2024-09-11T01:23:46.054414+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6816710,"num_objects":1666,"num_object_clones":0,"num_object_copies":3332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1666,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16956,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'18461","reported_seq":18632,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:46.106774+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:46.106774+0000","last_peered":"2024-09-11T01:23:46.106774+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:46.106774+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6488088,"num_objects":1585,"num_object_clones":0,"num_object_copies":3170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2378}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"44'18723","reported_seq":18772,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.169582+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:46.169582+0000","last_peered":"2024-09-11T01:23:46.169582+0000","last_clean":"2024-09-11T01:23:46.169582+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:46.169582+0000","last_undegraded":"2024-09-11T01:23:46.169582+0000","last_fullsized":"2024-09-11T01:23:46.169582+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6713344,"num_objects":1639,"num_object_clones":0,"num_object_copies":3278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18723,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'18574","reported_seq":18697,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:43.591420+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:43.591420+0000","last_peered":"2024-09-11T01:23:43.591420+0000","last_clean":"2024-09-11T01:23:43.591420+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:43.591420+0000","last_undegraded":"2024-09-11T01:23:43.591420+0000","last_fullsized":"2024-09-11T01:23:43.591420+0000","mapping_epoch":31,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6975971,"num_objects":1704,"num_object_clones":0,"num_object_copies":3408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15372,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'18585","reported_seq":18853,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.645523+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:43.645523+0000","last_peered":"2024-09-11T01:23:43.645523+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:43.645523+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"39'8500","ondisk_log_start":"39'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6644195,"num_objects":1623,"num_object_clones":0,"num_object_copies":3246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1623,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15073,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2426},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'19101","reported_seq":19384,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.683250+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:43.683250+0000","last_peered":"2024-09-11T01:23:43.683250+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:43.683250+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7009222,"num_objects":1713,"num_object_clones":0,"num_object_copies":3426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1713,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15854,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2493},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'18724","reported_seq":18845,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:23:43.622819+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:43.622819+0000","last_peered":"2024-09-11T01:23:43.622819+0000","last_clean":"2024-09-11T01:23:43.622819+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:43.622819+0000","last_undegraded":"2024-09-11T01:23:43.622819+0000","last_fullsized":"2024-09-11T01:23:43.622819+0000","mapping_epoch":31,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6963200,"num_objects":1700,"num_object_clones":0,"num_object_copies":3400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'18688","reported_seq":27036,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":40,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6394339,"num_objects":1562,"num_object_clones":0,"num_object_copies":3124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1562,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16555,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.840123+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:45.840123+0000","last_peered":"2024-09-11T01:23:45.840123+0000","last_clean":"2024-09-11T01:23:45.840123+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:45.840123+0000","last_undegraded":"2024-09-11T01:23:45.840123+0000","last_fullsized":"2024-09-11T01:23:45.840123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'18899","reported_seq":19180,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.645556+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:43.645556+0000","last_peered":"2024-09-11T01:23:43.645556+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:43.645556+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6942720,"num_objects":1695,"num_object_clones":0,"num_object_copies":3390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15263,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2335}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":99,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.671694+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:20:45.671694+0000","last_peered":"2024-09-11T01:20:45.671694+0000","last_clean":"2024-09-11T01:20:45.671694+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:20:45.671694+0000","last_undegraded":"2024-09-11T01:20:45.671694+0000","last_fullsized":"2024-09-11T01:20:45.671694+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'18461","reported_seq":26615,"reported_epoch":43,"state":"peering","last_fresh":"2024-09-11T01:23:45.838894+0000","last_change":"2024-09-11T01:23:45.850699+0000","last_active":"2024-09-11T01:23:45.850699+0000","last_peered":"2024-09-11T01:23:45.838894+0000","last_clean":"2024-09-11T01:15:17.951087+0000","last_became_active":"2024-09-11T01:15:20.508900+0000","last_became_peered":"2024-09-11T01:15:20.508900+0000","last_unstale":"2024-09-11T01:23:45.838894+0000","last_undegraded":"2024-09-11T01:23:45.838894+0000","last_fullsized":"2024-09-11T01:15:19.986999+0000","mapping_epoch":44,"log_start":"40'8400","ondisk_log_start":"40'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6873571,"num_objects":1679,"num_object_clones":0,"num_object_copies":3358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16230,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'18972","reported_seq":19129,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:46.169429+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:46.169429+0000","last_peered":"2024-09-11T01:23:46.169429+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:46.169429+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6512640,"num_objects":1590,"num_object_clones":0,"num_object_copies":3180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16609,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'18782","reported_seq":27258,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843202+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:46.843202+0000","last_peered":"2024-09-11T01:23:46.843202+0000","last_clean":"2024-09-11T01:23:46.843202+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:46.843202+0000","last_undegraded":"2024-09-11T01:23:46.843202+0000","last_fullsized":"2024-09-11T01:23:46.843202+0000","mapping_epoch":42,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6430720,"num_objects":1570,"num_object_clones":0,"num_object_copies":3140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15472,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'19014","reported_seq":19300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.591074+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:43.591074+0000","last_peered":"2024-09-11T01:23:43.591074+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:43.591074+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6684672,"num_objects":1632,"num_object_clones":0,"num_object_copies":3264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15407,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.843386+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:45.843386+0000","last_peered":"2024-09-11T01:23:45.843386+0000","last_clean":"2024-09-11T01:23:45.843386+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:45.843386+0000","last_undegraded":"2024-09-11T01:23:45.843386+0000","last_fullsized":"2024-09-11T01:23:45.843386+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'18847","reported_seq":19146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.639555+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:43.639555+0000","last_peered":"2024-09-11T01:23:43.639555+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:43.639555+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7180288,"num_objects":1753,"num_object_clones":0,"num_object_copies":3506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15382,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'18490","reported_seq":18731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:43.664106+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:43.664106+0000","last_peered":"2024-09-11T01:23:43.664106+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:43.664106+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"39'8400","ondisk_log_start":"39'8400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6751174,"num_objects":1650,"num_object_clones":0,"num_object_copies":3300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15147,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":108413686,"num_objects":26478,"num_object_clones":0,"num_object_copies":52956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36740,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26478,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":258763,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12894,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":418045952,"data_stored":417893288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510092,"kb_used_data":185464,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93861748,"statfs":{"total":96636764160,"available":96114429952,"internally_reserved":0,"allocated":189915136,"data_stored":189628068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":332394158},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790054,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537756,"kb_used_data":77064,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93834084,"statfs":{"total":96636764160,"available":96086102016,"internally_reserved":0,"allocated":78913536,"data_stored":78586261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572636,"kb_used_data":148084,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93799204,"statfs":{"total":96636764160,"available":96050384896,"internally_reserved":0,"allocated":151638016,"data_stored":151356183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":150061056,"data_stored":150024926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78467072,"data_stored":78390740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":189517824,"data_stored":189477622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:52.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:52.664 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:52.667 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:52.697 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-11T01:23:52.634038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":99779337,"num_objects":24372,"num_object_clones":0,"num_object_copies":48744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36622,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24372,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":260873,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13013,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1616492,"kb_used_data":394148,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281499028,"statfs":{"total":289910292480,"available":288255004672,"internally_reserved":0,"allocated":403607552,"data_stored":402690353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1251593223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,7,11,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12595200,"num_objects":-3075,"num_object_clones":0,"num_object_copies":-6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3075,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001910"},"pg_stats":[{"pgid":"2.f","version":"45'18657","reported_seq":19123,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.647038+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:48.647038+0000","last_peered":"2024-09-11T01:23:48.647038+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:48.647038+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6475776,"num_objects":1581,"num_object_clones":0,"num_object_copies":3162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18657,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'18991","reported_seq":19077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.110212+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:51.110212+0000","last_peered":"2024-09-11T01:23:51.110212+0000","last_clean":"2024-09-11T01:23:51.110212+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:51.110212+0000","last_undegraded":"2024-09-11T01:23:51.110212+0000","last_fullsized":"2024-09-11T01:23:51.110212+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271942,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17089,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18587","reported_seq":18759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:51.203320+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:51.203320+0000","last_peered":"2024-09-11T01:23:51.203320+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:51.203320+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8500","ondisk_log_start":"41'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5971992,"num_objects":1459,"num_object_clones":0,"num_object_copies":2918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16648,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'18851","reported_seq":18902,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.205661+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:51.205661+0000","last_peered":"2024-09-11T01:23:51.205661+0000","last_clean":"2024-09-11T01:23:51.205661+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:51.205661+0000","last_undegraded":"2024-09-11T01:23:51.205661+0000","last_fullsized":"2024-09-11T01:23:51.205661+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18851,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18700","reported_seq":18827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:48.600203+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:48.600203+0000","last_peered":"2024-09-11T01:23:48.600203+0000","last_clean":"2024-09-11T01:23:48.600203+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:48.600203+0000","last_undegraded":"2024-09-11T01:23:48.600203+0000","last_fullsized":"2024-09-11T01:23:48.600203+0000","mapping_epoch":31,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6459875,"num_objects":1578,"num_object_clones":0,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15498,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18697","reported_seq":18969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.659707+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:48.659707+0000","last_peered":"2024-09-11T01:23:48.659707+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:48.659707+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6185443,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15185,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2538},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19222","reported_seq":19515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.645367+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:48.645367+0000","last_peered":"2024-09-11T01:23:48.645367+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:48.645367+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513606,"num_objects":1592,"num_object_clones":0,"num_object_copies":3184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1592,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15975,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2614},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'18859","reported_seq":18984,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:48.656250+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:48.656250+0000","last_peered":"2024-09-11T01:23:48.656250+0000","last_clean":"2024-09-11T01:23:48.656250+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:48.656250+0000","last_undegraded":"2024-09-11T01:23:48.656250+0000","last_fullsized":"2024-09-11T01:23:48.656250+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6410240,"num_objects":1565,"num_object_clones":0,"num_object_copies":3130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15812,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'18819","reported_seq":27167,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.914548+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:51.914548+0000","last_peered":"2024-09-11T01:23:51.914548+0000","last_clean":"2024-09-11T01:23:51.914548+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:51.914548+0000","last_undegraded":"2024-09-11T01:23:51.914548+0000","last_fullsized":"2024-09-11T01:23:51.914548+0000","mapping_epoch":40,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857763,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19024","reported_seq":19309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.620571+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:48.620571+0000","last_peered":"2024-09-11T01:23:48.620571+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:48.620571+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6430720,"num_objects":1570,"num_object_clones":0,"num_object_copies":3140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15388,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18608","reported_seq":26771,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.195769+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:23:51.195769+0000","last_peered":"2024-09-11T01:23:51.195769+0000","last_clean":"2024-09-11T01:23:51.195769+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:23:51.195769+0000","last_undegraded":"2024-09-11T01:23:51.195769+0000","last_fullsized":"2024-09-11T01:23:51.195769+0000","mapping_epoch":44,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271459,"num_objects":1532,"num_object_clones":0,"num_object_copies":3064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16377,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19107","reported_seq":19266,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:51.084355+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:51.084355+0000","last_peered":"2024-09-11T01:23:51.084355+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:51.084355+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5959680,"num_objects":1455,"num_object_clones":0,"num_object_copies":2910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16744,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'18927","reported_seq":27403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.986421+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:51.986421+0000","last_peered":"2024-09-11T01:23:51.986421+0000","last_clean":"2024-09-11T01:23:51.986421+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:51.986421+0000","last_undegraded":"2024-09-11T01:23:51.986421+0000","last_fullsized":"2024-09-11T01:23:51.986421+0000","mapping_epoch":42,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5836800,"num_objects":1425,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19153","reported_seq":19441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.663759+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:48.663759+0000","last_peered":"2024-09-11T01:23:48.663759+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:48.663759+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6115328,"num_objects":1493,"num_object_clones":0,"num_object_copies":2986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15546,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'18991","reported_seq":19296,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.620620+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:48.620620+0000","last_peered":"2024-09-11T01:23:48.620620+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:48.620620+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6590464,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15526,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18615","reported_seq":18859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.594892+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:48.594892+0000","last_peered":"2024-09-11T01:23:48.594892+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:48.594892+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6239174,"num_objects":1525,"num_object_clones":0,"num_object_copies":3050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15272,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2577},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":99779318,"num_objects":24370,"num_object_clones":0,"num_object_copies":48740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36622,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24370,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":260871,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13012,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":400146432,"data_stored":399993768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510188,"kb_used_data":181404,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93861652,"statfs":{"total":96636764160,"available":96114331648,"internally_reserved":0,"allocated":185757696,"data_stored":185465367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790055,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535628,"kb_used_data":70780,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93836212,"statfs":{"total":96636764160,"available":96088281088,"internally_reserved":0,"allocated":72478720,"data_stored":72146184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570676,"kb_used_data":141964,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93801164,"statfs":{"total":96636764160,"available":96052391936,"internally_reserved":0,"allocated":145371136,"data_stored":145078802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143675392,"data_stored":143639262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71266304,"data_stored":71189972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185204736,"data_stored":185164534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:52.699 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:53.165 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:53.168 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:53.187 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-11T01:23:52.634038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":99779337,"num_objects":24372,"num_object_clones":0,"num_object_copies":48744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36622,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24372,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":260873,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13013,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1616492,"kb_used_data":394148,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281499028,"statfs":{"total":289910292480,"available":288255004672,"internally_reserved":0,"allocated":403607552,"data_stored":402690353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1251593223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,7,11,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12595200,"num_objects":-3075,"num_object_clones":0,"num_object_copies":-6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3075,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001910"},"pg_stats":[{"pgid":"2.f","version":"45'18657","reported_seq":19123,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.647038+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:48.647038+0000","last_peered":"2024-09-11T01:23:48.647038+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:48.647038+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6475776,"num_objects":1581,"num_object_clones":0,"num_object_copies":3162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18657,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'18991","reported_seq":19077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.110212+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:51.110212+0000","last_peered":"2024-09-11T01:23:51.110212+0000","last_clean":"2024-09-11T01:23:51.110212+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:51.110212+0000","last_undegraded":"2024-09-11T01:23:51.110212+0000","last_fullsized":"2024-09-11T01:23:51.110212+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271942,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17089,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18587","reported_seq":18759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:51.203320+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:51.203320+0000","last_peered":"2024-09-11T01:23:51.203320+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:51.203320+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8500","ondisk_log_start":"41'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5971992,"num_objects":1459,"num_object_clones":0,"num_object_copies":2918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16648,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'18851","reported_seq":18902,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.205661+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:51.205661+0000","last_peered":"2024-09-11T01:23:51.205661+0000","last_clean":"2024-09-11T01:23:51.205661+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:51.205661+0000","last_undegraded":"2024-09-11T01:23:51.205661+0000","last_fullsized":"2024-09-11T01:23:51.205661+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18851,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18700","reported_seq":18827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:48.600203+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:48.600203+0000","last_peered":"2024-09-11T01:23:48.600203+0000","last_clean":"2024-09-11T01:23:48.600203+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:48.600203+0000","last_undegraded":"2024-09-11T01:23:48.600203+0000","last_fullsized":"2024-09-11T01:23:48.600203+0000","mapping_epoch":31,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6459875,"num_objects":1578,"num_object_clones":0,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15498,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18697","reported_seq":18969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.659707+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:48.659707+0000","last_peered":"2024-09-11T01:23:48.659707+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:48.659707+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6185443,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15185,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2538},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19222","reported_seq":19515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.645367+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:48.645367+0000","last_peered":"2024-09-11T01:23:48.645367+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:48.645367+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513606,"num_objects":1592,"num_object_clones":0,"num_object_copies":3184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1592,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15975,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2614},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'18859","reported_seq":18984,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:48.656250+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:48.656250+0000","last_peered":"2024-09-11T01:23:48.656250+0000","last_clean":"2024-09-11T01:23:48.656250+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:48.656250+0000","last_undegraded":"2024-09-11T01:23:48.656250+0000","last_fullsized":"2024-09-11T01:23:48.656250+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6410240,"num_objects":1565,"num_object_clones":0,"num_object_copies":3130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15812,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'18819","reported_seq":27167,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.914548+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:51.914548+0000","last_peered":"2024-09-11T01:23:51.914548+0000","last_clean":"2024-09-11T01:23:51.914548+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:51.914548+0000","last_undegraded":"2024-09-11T01:23:51.914548+0000","last_fullsized":"2024-09-11T01:23:51.914548+0000","mapping_epoch":40,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857763,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19024","reported_seq":19309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.620571+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:48.620571+0000","last_peered":"2024-09-11T01:23:48.620571+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:48.620571+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6430720,"num_objects":1570,"num_object_clones":0,"num_object_copies":3140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15388,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18608","reported_seq":26771,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.195769+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:23:51.195769+0000","last_peered":"2024-09-11T01:23:51.195769+0000","last_clean":"2024-09-11T01:23:51.195769+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:23:51.195769+0000","last_undegraded":"2024-09-11T01:23:51.195769+0000","last_fullsized":"2024-09-11T01:23:51.195769+0000","mapping_epoch":44,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271459,"num_objects":1532,"num_object_clones":0,"num_object_copies":3064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16377,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19107","reported_seq":19266,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:51.084355+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:51.084355+0000","last_peered":"2024-09-11T01:23:51.084355+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:51.084355+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5959680,"num_objects":1455,"num_object_clones":0,"num_object_copies":2910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16744,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'18927","reported_seq":27403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.986421+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:51.986421+0000","last_peered":"2024-09-11T01:23:51.986421+0000","last_clean":"2024-09-11T01:23:51.986421+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:51.986421+0000","last_undegraded":"2024-09-11T01:23:51.986421+0000","last_fullsized":"2024-09-11T01:23:51.986421+0000","mapping_epoch":42,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5836800,"num_objects":1425,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19153","reported_seq":19441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.663759+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:48.663759+0000","last_peered":"2024-09-11T01:23:48.663759+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:48.663759+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6115328,"num_objects":1493,"num_object_clones":0,"num_object_copies":2986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15546,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'18991","reported_seq":19296,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.620620+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:48.620620+0000","last_peered":"2024-09-11T01:23:48.620620+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:48.620620+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6590464,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15526,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18615","reported_seq":18859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.594892+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:48.594892+0000","last_peered":"2024-09-11T01:23:48.594892+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:48.594892+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6239174,"num_objects":1525,"num_object_clones":0,"num_object_copies":3050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15272,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2577},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":99779318,"num_objects":24370,"num_object_clones":0,"num_object_copies":48740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36622,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24370,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":260871,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13012,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":400146432,"data_stored":399993768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510188,"kb_used_data":181404,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93861652,"statfs":{"total":96636764160,"available":96114331648,"internally_reserved":0,"allocated":185757696,"data_stored":185465367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790055,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535628,"kb_used_data":70780,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93836212,"statfs":{"total":96636764160,"available":96088281088,"internally_reserved":0,"allocated":72478720,"data_stored":72146184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570676,"kb_used_data":141964,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93801164,"statfs":{"total":96636764160,"available":96052391936,"internally_reserved":0,"allocated":145371136,"data_stored":145078802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143675392,"data_stored":143639262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71266304,"data_stored":71189972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185204736,"data_stored":185164534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:53.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:53.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'18657", 'reported_seq': 19123, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.647038+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:23:48.647038+0000', 'last_peered': '2024-09-11T01:23:48.647038+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:48.647038+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'8600", 'ondisk_log_start': "41'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6475776, 'num_objects': 1581, 'num_object_clones': 0, 'num_object_copies': 3162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3930, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18657, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 193, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2349}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:53.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'18587", 'reported_seq': 18759, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:51.203320+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:51.203320+0000', 'last_peered': '2024-09-11T01:23:51.203320+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:51.203320+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'8500", 'ondisk_log_start': "41'8500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5971992, 'num_objects': 1459, 'num_object_clones': 0, 'num_object_copies': 2918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1459, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16648, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2504}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:53.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:53.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'18697", 'reported_seq': 18969, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.659707+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:48.659707+0000', 'last_peered': '2024-09-11T01:23:48.659707+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:48.659707+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'8600", 'ondisk_log_start': "41'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6185443, 'num_objects': 1511, 'num_object_clones': 0, 'num_object_copies': 3022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1511, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15185, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2538}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:53.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'19222", 'reported_seq': 19515, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.645367+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:48.645367+0000', 'last_peered': '2024-09-11T01:23:48.645367+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:48.645367+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'9200", 'ondisk_log_start': "41'9200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6513606, 'num_objects': 1592, 'num_object_clones': 0, 'num_object_copies': 3184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1592, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 15975, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2614}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:53.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'19024", 'reported_seq': 19309, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.620571+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:48.620571+0000', 'last_peered': '2024-09-11T01:23:48.620571+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:48.620571+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'9000", 'ondisk_log_start': "41'9000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6430720, 'num_objects': 1570, 'num_object_clones': 0, 'num_object_copies': 3140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15388, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2460}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:53.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'19107", 'reported_seq': 19266, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:51.084355+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:51.084355+0000', 'last_peered': '2024-09-11T01:23:51.084355+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:51.084355+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5959680, 'num_objects': 1455, 'num_object_clones': 0, 'num_object_copies': 2910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16744, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2692}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:53.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:53.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'19153", 'reported_seq': 19441, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.663759+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:48.663759+0000', 'last_peered': '2024-09-11T01:23:48.663759+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:48.663759+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6115328, 'num_objects': 1493, 'num_object_clones': 0, 'num_object_copies': 2986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15546, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2624}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:53.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'18991", 'reported_seq': 19296, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.620620+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:48.620620+0000', 'last_peered': '2024-09-11T01:23:48.620620+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:48.620620+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'8900", 'ondisk_log_start': "41'8900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6590464, 'num_objects': 1609, 'num_object_clones': 0, 'num_object_copies': 3218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1609, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15526, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2456}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.193 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:53.193 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'18615", 'reported_seq': 18859, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:48.594892+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:48.594892+0000', 'last_peered': '2024-09-11T01:23:48.594892+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:48.594892+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'8600", 'ondisk_log_start': "41'8600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6239174, 'num_objects': 1525, 'num_object_clones': 0, 'num_object_copies': 3050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1525, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15272, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2577}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:53.193 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:53.949 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:53.949 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36622/48744 objects degraded (75.131%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":24372,"data_bytes":99779337,"bytes_used":1655287808,"bytes_avail":288255004672,"bytes_total":289910292480,"degraded_objects":36622,"degraded_total":48744,"degraded_ratio":0.7513129821106187,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:53.981 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:53.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:54.561 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:54.563 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:54.583 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-11T01:23:52.634038+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":99779337,"num_objects":24372,"num_object_clones":0,"num_object_copies":48744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36622,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24372,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":260873,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13013,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1616492,"kb_used_data":394148,"kb_used_omap":12,"kb_used_meta":1222259,"kb_avail":281499028,"statfs":{"total":289910292480,"available":288255004672,"internally_reserved":0,"allocated":403607552,"data_stored":402690353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1251593223},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,7,11,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12595200,"num_objects":-3075,"num_object_clones":0,"num_object_copies":-6150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3075,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001910"},"pg_stats":[{"pgid":"2.f","version":"45'18657","reported_seq":19123,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.647038+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:48.647038+0000","last_peered":"2024-09-11T01:23:48.647038+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:48.647038+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6475776,"num_objects":1581,"num_object_clones":0,"num_object_copies":3162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18657,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'18991","reported_seq":19077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.110212+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:51.110212+0000","last_peered":"2024-09-11T01:23:51.110212+0000","last_clean":"2024-09-11T01:23:51.110212+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:51.110212+0000","last_undegraded":"2024-09-11T01:23:51.110212+0000","last_fullsized":"2024-09-11T01:23:51.110212+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271942,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17089,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18587","reported_seq":18759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:51.203320+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:51.203320+0000","last_peered":"2024-09-11T01:23:51.203320+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:51.203320+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8500","ondisk_log_start":"41'8500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5971992,"num_objects":1459,"num_object_clones":0,"num_object_copies":2918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16648,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'18851","reported_seq":18902,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.205661+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:51.205661+0000","last_peered":"2024-09-11T01:23:51.205661+0000","last_clean":"2024-09-11T01:23:51.205661+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:51.205661+0000","last_undegraded":"2024-09-11T01:23:51.205661+0000","last_fullsized":"2024-09-11T01:23:51.205661+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18851,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18700","reported_seq":18827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:48.600203+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:48.600203+0000","last_peered":"2024-09-11T01:23:48.600203+0000","last_clean":"2024-09-11T01:23:48.600203+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:48.600203+0000","last_undegraded":"2024-09-11T01:23:48.600203+0000","last_fullsized":"2024-09-11T01:23:48.600203+0000","mapping_epoch":31,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6459875,"num_objects":1578,"num_object_clones":0,"num_object_copies":3156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15498,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18697","reported_seq":18969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.659707+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:48.659707+0000","last_peered":"2024-09-11T01:23:48.659707+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:48.659707+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6185443,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15185,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2538},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19222","reported_seq":19515,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.645367+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:48.645367+0000","last_peered":"2024-09-11T01:23:48.645367+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:48.645367+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6513606,"num_objects":1592,"num_object_clones":0,"num_object_copies":3184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1592,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15975,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2614},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'18859","reported_seq":18984,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:48.656250+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:48.656250+0000","last_peered":"2024-09-11T01:23:48.656250+0000","last_clean":"2024-09-11T01:23:48.656250+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:48.656250+0000","last_undegraded":"2024-09-11T01:23:48.656250+0000","last_fullsized":"2024-09-11T01:23:48.656250+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6410240,"num_objects":1565,"num_object_clones":0,"num_object_copies":3130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15812,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'18819","reported_seq":27167,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.914548+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:51.914548+0000","last_peered":"2024-09-11T01:23:51.914548+0000","last_clean":"2024-09-11T01:23:51.914548+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:51.914548+0000","last_undegraded":"2024-09-11T01:23:51.914548+0000","last_fullsized":"2024-09-11T01:23:51.914548+0000","mapping_epoch":40,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857763,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19024","reported_seq":19309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.620571+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:48.620571+0000","last_peered":"2024-09-11T01:23:48.620571+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:48.620571+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6430720,"num_objects":1570,"num_object_clones":0,"num_object_copies":3140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15388,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18608","reported_seq":26771,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.195769+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:23:51.195769+0000","last_peered":"2024-09-11T01:23:51.195769+0000","last_clean":"2024-09-11T01:23:51.195769+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:23:51.195769+0000","last_undegraded":"2024-09-11T01:23:51.195769+0000","last_fullsized":"2024-09-11T01:23:51.195769+0000","mapping_epoch":44,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6271459,"num_objects":1532,"num_object_clones":0,"num_object_copies":3064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1532,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16377,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19107","reported_seq":19266,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:51.084355+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:51.084355+0000","last_peered":"2024-09-11T01:23:51.084355+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:51.084355+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5959680,"num_objects":1455,"num_object_clones":0,"num_object_copies":2910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16744,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2692}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'18927","reported_seq":27403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.986421+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:51.986421+0000","last_peered":"2024-09-11T01:23:51.986421+0000","last_clean":"2024-09-11T01:23:51.986421+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:51.986421+0000","last_undegraded":"2024-09-11T01:23:51.986421+0000","last_fullsized":"2024-09-11T01:23:51.986421+0000","mapping_epoch":42,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5836800,"num_objects":1425,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19153","reported_seq":19441,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.663759+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:48.663759+0000","last_peered":"2024-09-11T01:23:48.663759+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:48.663759+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6115328,"num_objects":1493,"num_object_clones":0,"num_object_copies":2986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15546,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2624}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'18991","reported_seq":19296,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.620620+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:48.620620+0000","last_peered":"2024-09-11T01:23:48.620620+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:48.620620+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6590464,"num_objects":1609,"num_object_clones":0,"num_object_copies":3218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15526,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2456}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18615","reported_seq":18859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:48.594892+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:48.594892+0000","last_peered":"2024-09-11T01:23:48.594892+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:48.594892+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8600","ondisk_log_start":"41'8600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6239174,"num_objects":1525,"num_object_clones":0,"num_object_copies":3050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15272,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2577},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":99779318,"num_objects":24370,"num_object_clones":0,"num_object_copies":48740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36622,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24370,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":260871,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13012,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":400146432,"data_stored":399993768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510188,"kb_used_data":181404,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93861652,"statfs":{"total":96636764160,"available":96114331648,"internally_reserved":0,"allocated":185757696,"data_stored":185465367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":336653998},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790055,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535628,"kb_used_data":70780,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93836212,"statfs":{"total":96636764160,"available":96088281088,"internally_reserved":0,"allocated":72478720,"data_stored":72146184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570676,"kb_used_data":141964,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93801164,"statfs":{"total":96636764160,"available":96052391936,"internally_reserved":0,"allocated":145371136,"data_stored":145078802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":438955690},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":143675392,"data_stored":143639262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71266304,"data_stored":71189972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":185204736,"data_stored":185164534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:57.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:58.189 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:58.191 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:58.220 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-11T01:23:56.637544+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":91726601,"num_objects":22406,"num_object_clones":0,"num_object_copies":44812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22406,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":262839,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13102,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160877,"ondisk_log_size":160877,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1600864,"kb_used_data":382704,"kb_used_omap":12,"kb_used_meta":1218099,"kb_avail":281514656,"statfs":{"total":289910292480,"available":288271007744,"internally_reserved":0,"allocated":391888896,"data_stored":390971697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1247333383},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,6,2,8,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17694720,"num_objects":-4320,"num_object_clones":0,"num_object_copies":-8640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004962"},"pg_stats":[{"pgid":"2.f","version":"45'18795","reported_seq":19439,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.639648+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:53.639648+0000","last_peered":"2024-09-11T01:23:53.639648+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:53.639648+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5910528,"num_objects":1443,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18795,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19138","reported_seq":19224,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.226195+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:56.226195+0000","last_peered":"2024-09-11T01:23:56.226195+0000","last_clean":"2024-09-11T01:23:56.226195+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:56.226195+0000","last_undegraded":"2024-09-11T01:23:56.226195+0000","last_fullsized":"2024-09-11T01:23:56.226195+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5669830,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17236,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18720","reported_seq":18889,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:56.177975+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:56.177975+0000","last_peered":"2024-09-11T01:23:56.177975+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:56.177975+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5427224,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16781,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'18998","reported_seq":19049,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.184212+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:56.184212+0000","last_peered":"2024-09-11T01:23:56.184212+0000","last_clean":"2024-09-11T01:23:56.184212+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:56.184212+0000","last_undegraded":"2024-09-11T01:23:56.184212+0000","last_fullsized":"2024-09-11T01:23:56.184212+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18998,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18827","reported_seq":18954,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:53.682815+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:53.682815+0000","last_peered":"2024-09-11T01:23:53.682815+0000","last_clean":"2024-09-11T01:23:53.682815+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:53.682815+0000","last_undegraded":"2024-09-11T01:23:53.682815+0000","last_fullsized":"2024-09-11T01:23:53.682815+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5939683,"num_objects":1451,"num_object_clones":0,"num_object_copies":2902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15625,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18845","reported_seq":19117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.583590+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:53.583590+0000","last_peered":"2024-09-11T01:23:53.583590+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:53.583590+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5579235,"num_objects":1363,"num_object_clones":0,"num_object_copies":2726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15333,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2686},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19377","reported_seq":19668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.691637+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:53.691637+0000","last_peered":"2024-09-11T01:23:53.691637+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:53.691637+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5878726,"num_objects":1437,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16130,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2769},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'18992","reported_seq":19117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:53.668640+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:53.668640+0000","last_peered":"2024-09-11T01:23:53.668640+0000","last_clean":"2024-09-11T01:23:53.668640+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:53.668640+0000","last_undegraded":"2024-09-11T01:23:53.668640+0000","last_fullsized":"2024-09-11T01:23:53.668640+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865472,"num_objects":1432,"num_object_clones":0,"num_object_copies":2864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'18819","reported_seq":27167,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.914548+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:51.914548+0000","last_peered":"2024-09-11T01:23:51.914548+0000","last_clean":"2024-09-11T01:23:51.914548+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:51.914548+0000","last_undegraded":"2024-09-11T01:23:51.914548+0000","last_fullsized":"2024-09-11T01:23:51.914548+0000","mapping_epoch":40,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5857763,"num_objects":1431,"num_object_clones":0,"num_object_copies":2862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1431,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16686,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19157","reported_seq":19442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.700998+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:53.700998+0000","last_peered":"2024-09-11T01:23:53.700998+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:53.700998+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5885952,"num_objects":1437,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2593}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18753","reported_seq":26916,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.213512+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:23:56.213512+0000","last_peered":"2024-09-11T01:23:56.213512+0000","last_clean":"2024-09-11T01:23:56.213512+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:23:56.213512+0000","last_undegraded":"2024-09-11T01:23:56.213512+0000","last_fullsized":"2024-09-11T01:23:56.213512+0000","mapping_epoch":44,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677539,"num_objects":1387,"num_object_clones":0,"num_object_copies":2774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1387,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19235","reported_seq":19394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:56.172865+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:56.172865+0000","last_peered":"2024-09-11T01:23:56.172865+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:56.172865+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435392,"num_objects":1327,"num_object_clones":0,"num_object_copies":2654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2820}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'18927","reported_seq":27403,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:51.986421+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:51.986421+0000","last_peered":"2024-09-11T01:23:51.986421+0000","last_clean":"2024-09-11T01:23:51.986421+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:51.986421+0000","last_undegraded":"2024-09-11T01:23:51.986421+0000","last_fullsized":"2024-09-11T01:23:51.986421+0000","mapping_epoch":42,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5836800,"num_objects":1425,"num_object_clones":0,"num_object_copies":2850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15617,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19287","reported_seq":19576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.551182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:53.551182+0000","last_peered":"2024-09-11T01:23:53.551182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:53.551182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5566464,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19143","reported_seq":19448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.530793+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:53.530793+0000","last_peered":"2024-09-11T01:23:53.530793+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:53.530793+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5967872,"num_objects":1457,"num_object_clones":0,"num_object_copies":2914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18761","reported_seq":19006,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.598662+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:53.598662+0000","last_peered":"2024-09-11T01:23:53.598662+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:53.598662+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5641158,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15418,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2723},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":91726582,"num_objects":22404,"num_object_clones":0,"num_object_copies":44808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22404,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":262837,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13101,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":388628480,"data_stored":388475816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160874,"ondisk_log_size":160874,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493328,"kb_used_data":177024,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93878512,"statfs":{"total":96636764160,"available":96131596288,"internally_reserved":0,"allocated":181272576,"data_stored":180980247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790055,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535628,"kb_used_data":70780,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93836212,"statfs":{"total":96636764160,"available":96088281088,"internally_reserved":0,"allocated":72478720,"data_stored":72146184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,1,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571908,"kb_used_data":134900,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93799932,"statfs":{"total":96636764160,"available":96051130368,"internally_reserved":0,"allocated":138137600,"data_stored":137845266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136654848,"data_stored":136618718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71266304,"data_stored":71189972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":180707328,"data_stored":180667126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:58.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:58.848 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:58.850 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:23:58.878 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-11T01:23:58.638011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90555145,"num_objects":22120,"num_object_clones":0,"num_object_copies":44240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22120,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":263125,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13102,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160963,"ondisk_log_size":160963,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1602624,"kb_used_data":376144,"kb_used_omap":12,"kb_used_meta":1226419,"kb_avail":281512896,"statfs":{"total":289910292480,"available":288269205504,"internally_reserved":0,"allocated":385171456,"data_stored":384254257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1255853063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,3,8,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18866176,"num_objects":-4606,"num_object_clones":0,"num_object_copies":-9212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4606,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.799635"},"pg_stats":[{"pgid":"2.f","version":"45'18795","reported_seq":19439,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.639648+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:53.639648+0000","last_peered":"2024-09-11T01:23:53.639648+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:53.639648+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5910528,"num_objects":1443,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18795,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19138","reported_seq":19224,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.226195+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:56.226195+0000","last_peered":"2024-09-11T01:23:56.226195+0000","last_clean":"2024-09-11T01:23:56.226195+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:56.226195+0000","last_undegraded":"2024-09-11T01:23:56.226195+0000","last_fullsized":"2024-09-11T01:23:56.226195+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5669830,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17236,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18720","reported_seq":18889,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:56.177975+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:56.177975+0000","last_peered":"2024-09-11T01:23:56.177975+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:56.177975+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5427224,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16781,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'18998","reported_seq":19049,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.184212+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:56.184212+0000","last_peered":"2024-09-11T01:23:56.184212+0000","last_clean":"2024-09-11T01:23:56.184212+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:56.184212+0000","last_undegraded":"2024-09-11T01:23:56.184212+0000","last_fullsized":"2024-09-11T01:23:56.184212+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18998,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18827","reported_seq":18954,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:53.682815+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:53.682815+0000","last_peered":"2024-09-11T01:23:53.682815+0000","last_clean":"2024-09-11T01:23:53.682815+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:53.682815+0000","last_undegraded":"2024-09-11T01:23:53.682815+0000","last_fullsized":"2024-09-11T01:23:53.682815+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5939683,"num_objects":1451,"num_object_clones":0,"num_object_copies":2902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15625,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18845","reported_seq":19117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.583590+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:53.583590+0000","last_peered":"2024-09-11T01:23:53.583590+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:53.583590+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5579235,"num_objects":1363,"num_object_clones":0,"num_object_copies":2726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15333,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2686},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19377","reported_seq":19668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.691637+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:53.691637+0000","last_peered":"2024-09-11T01:23:53.691637+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:53.691637+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5878726,"num_objects":1437,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16130,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2769},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'18992","reported_seq":19117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:53.668640+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:53.668640+0000","last_peered":"2024-09-11T01:23:53.668640+0000","last_clean":"2024-09-11T01:23:53.668640+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:53.668640+0000","last_undegraded":"2024-09-11T01:23:53.668640+0000","last_fullsized":"2024-09-11T01:23:53.668640+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865472,"num_objects":1432,"num_object_clones":0,"num_object_copies":2864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'18963","reported_seq":27312,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:57.002917+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:57.002917+0000","last_peered":"2024-09-11T01:23:57.002917+0000","last_clean":"2024-09-11T01:23:57.002917+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:57.002917+0000","last_undegraded":"2024-09-11T01:23:57.002917+0000","last_fullsized":"2024-09-11T01:23:57.002917+0000","mapping_epoch":40,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267939,"num_objects":1287,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16830,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19157","reported_seq":19442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.700998+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:53.700998+0000","last_peered":"2024-09-11T01:23:53.700998+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:53.700998+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5885952,"num_objects":1437,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2593}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18753","reported_seq":26916,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.213512+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:23:56.213512+0000","last_peered":"2024-09-11T01:23:56.213512+0000","last_clean":"2024-09-11T01:23:56.213512+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:23:56.213512+0000","last_undegraded":"2024-09-11T01:23:56.213512+0000","last_fullsized":"2024-09-11T01:23:56.213512+0000","mapping_epoch":44,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677539,"num_objects":1387,"num_object_clones":0,"num_object_copies":2774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1387,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19235","reported_seq":19394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:56.172865+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:56.172865+0000","last_peered":"2024-09-11T01:23:56.172865+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:56.172865+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435392,"num_objects":1327,"num_object_clones":0,"num_object_copies":2654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2820}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19069","reported_seq":27544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.972373+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:56.972373+0000","last_peered":"2024-09-11T01:23:56.972373+0000","last_clean":"2024-09-11T01:23:56.972373+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:56.972373+0000","last_undegraded":"2024-09-11T01:23:56.972373+0000","last_fullsized":"2024-09-11T01:23:56.972373+0000","mapping_epoch":42,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5255168,"num_objects":1283,"num_object_clones":0,"num_object_copies":2566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15759,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19287","reported_seq":19576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.551182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:53.551182+0000","last_peered":"2024-09-11T01:23:53.551182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:53.551182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5566464,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19143","reported_seq":19448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.530793+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:53.530793+0000","last_peered":"2024-09-11T01:23:53.530793+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:53.530793+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5967872,"num_objects":1457,"num_object_clones":0,"num_object_copies":2914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18761","reported_seq":19006,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.598662+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:53.598662+0000","last_peered":"2024-09-11T01:23:53.598662+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:53.598662+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5641158,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15418,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2723},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":90555126,"num_objects":22118,"num_object_clones":0,"num_object_copies":44236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22118,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":263123,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13101,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381779968,"data_stored":381627304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160960,"ondisk_log_size":160960,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493328,"kb_used_data":177024,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93878512,"statfs":{"total":96636764160,"available":96131596288,"internally_reserved":0,"allocated":181272576,"data_stored":180980247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790056,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537388,"kb_used_data":64220,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93834452,"statfs":{"total":96636764160,"available":96086478848,"internally_reserved":0,"allocated":65761280,"data_stored":65428744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571908,"kb_used_data":134900,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93799932,"statfs":{"total":96636764160,"available":96051130368,"internally_reserved":0,"allocated":138137600,"data_stored":137845266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136654848,"data_stored":136618718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64417792,"data_stored":64341460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":180707328,"data_stored":180667126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:58.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:58.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'18795", 'reported_seq': 19439, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.639648+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:23:53.639648+0000', 'last_peered': '2024-09-11T01:23:53.639648+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:53.639648+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'8700", 'ondisk_log_start': "41'8700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5910528, 'num_objects': 1443, 'num_object_clones': 0, 'num_object_copies': 2886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3841, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18795, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 282, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2398}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:23:58.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'18720", 'reported_seq': 18889, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:56.177975+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:23:56.177975+0000', 'last_peered': '2024-09-11T01:23:56.177975+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:23:56.177975+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'8700", 'ondisk_log_start': "41'8700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5427224, 'num_objects': 1326, 'num_object_clones': 0, 'num_object_copies': 2652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1326, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16781, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2637}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:58.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:58.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'18845", 'reported_seq': 19117, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.583590+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:53.583590+0000', 'last_peered': '2024-09-11T01:23:53.583590+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:53.583590+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'8800", 'ondisk_log_start': "41'8800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5579235, 'num_objects': 1363, 'num_object_clones': 0, 'num_object_copies': 2726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1363, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15333, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2686}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'19377", 'reported_seq': 19668, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.691637+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:53.691637+0000', 'last_peered': '2024-09-11T01:23:53.691637+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:53.691637+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'9300", 'ondisk_log_start': "41'9300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5878726, 'num_objects': 1437, 'num_object_clones': 0, 'num_object_copies': 2874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1437, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16130, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2769}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'19157", 'reported_seq': 19442, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.700998+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:53.700998+0000', 'last_peered': '2024-09-11T01:23:53.700998+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:53.700998+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5885952, 'num_objects': 1437, 'num_object_clones': 0, 'num_object_copies': 2874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15521, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2593}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'19235", 'reported_seq': 19394, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:56.172865+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:23:56.172865+0000', 'last_peered': '2024-09-11T01:23:56.172865+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:23:56.172865+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'9200", 'ondisk_log_start': "41'9200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5435392, 'num_objects': 1327, 'num_object_clones': 0, 'num_object_copies': 2654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1327, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16872, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2820}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'19287", 'reported_seq': 19576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.551182+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:53.551182+0000', 'last_peered': '2024-09-11T01:23:53.551182+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:53.551182+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9200", 'ondisk_log_start': "41'9200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5566464, 'num_objects': 1359, 'num_object_clones': 0, 'num_object_copies': 2718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15680, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'19143", 'reported_seq': 19448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.530793+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:53.530793+0000', 'last_peered': '2024-09-11T01:23:53.530793+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:53.530793+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5967872, 'num_objects': 1457, 'num_object_clones': 0, 'num_object_copies': 2914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15678, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:58.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'18761", 'reported_seq': 19006, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:53.598662+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:53.598662+0000', 'last_peered': '2024-09-11T01:23:53.598662+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:53.598662+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'8700", 'ondisk_log_start': "41'8700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5641158, 'num_objects': 1379, 'num_object_clones': 0, 'num_object_copies': 2758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1379, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15418, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2723}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:58.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:59.593 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:23:59.593 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36533/44240 objects degraded (82.579%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":640,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":22120,"data_bytes":90555145,"bytes_used":1641086976,"bytes_avail":288269205504,"bytes_total":289910292480,"degraded_objects":36533,"degraded_total":44240,"degraded_ratio":0.82579113924050629,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:59.622 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:59.623 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:00.212 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:00.215 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:00.237 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-11T01:23:58.638011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90555145,"num_objects":22120,"num_object_clones":0,"num_object_copies":44240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22120,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":263125,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13102,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160963,"ondisk_log_size":160963,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1602624,"kb_used_data":376144,"kb_used_omap":12,"kb_used_meta":1226419,"kb_avail":281512896,"statfs":{"total":289910292480,"available":288269205504,"internally_reserved":0,"allocated":385171456,"data_stored":384254257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1255853063},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,3,8,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18866176,"num_objects":-4606,"num_object_clones":0,"num_object_copies":-9212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4606,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.799635"},"pg_stats":[{"pgid":"2.f","version":"45'18795","reported_seq":19439,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.639648+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:53.639648+0000","last_peered":"2024-09-11T01:23:53.639648+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:53.639648+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5910528,"num_objects":1443,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18795,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2398}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19138","reported_seq":19224,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.226195+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:23:56.226195+0000","last_peered":"2024-09-11T01:23:56.226195+0000","last_clean":"2024-09-11T01:23:56.226195+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:23:56.226195+0000","last_undegraded":"2024-09-11T01:23:56.226195+0000","last_fullsized":"2024-09-11T01:23:56.226195+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5669830,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17236,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18720","reported_seq":18889,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:56.177975+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:23:56.177975+0000","last_peered":"2024-09-11T01:23:56.177975+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:23:56.177975+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5427224,"num_objects":1326,"num_object_clones":0,"num_object_copies":2652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16781,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'18998","reported_seq":19049,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.184212+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:23:56.184212+0000","last_peered":"2024-09-11T01:23:56.184212+0000","last_clean":"2024-09-11T01:23:56.184212+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:23:56.184212+0000","last_undegraded":"2024-09-11T01:23:56.184212+0000","last_fullsized":"2024-09-11T01:23:56.184212+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18998,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18827","reported_seq":18954,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:53.682815+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:53.682815+0000","last_peered":"2024-09-11T01:23:53.682815+0000","last_clean":"2024-09-11T01:23:53.682815+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:53.682815+0000","last_undegraded":"2024-09-11T01:23:53.682815+0000","last_fullsized":"2024-09-11T01:23:53.682815+0000","mapping_epoch":31,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5939683,"num_objects":1451,"num_object_clones":0,"num_object_copies":2902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15625,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18845","reported_seq":19117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.583590+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:53.583590+0000","last_peered":"2024-09-11T01:23:53.583590+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:53.583590+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5579235,"num_objects":1363,"num_object_clones":0,"num_object_copies":2726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15333,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2686},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19377","reported_seq":19668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.691637+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:53.691637+0000","last_peered":"2024-09-11T01:23:53.691637+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:53.691637+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5878726,"num_objects":1437,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16130,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2769},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'18992","reported_seq":19117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:53.668640+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:53.668640+0000","last_peered":"2024-09-11T01:23:53.668640+0000","last_clean":"2024-09-11T01:23:53.668640+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:53.668640+0000","last_undegraded":"2024-09-11T01:23:53.668640+0000","last_fullsized":"2024-09-11T01:23:53.668640+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5865472,"num_objects":1432,"num_object_clones":0,"num_object_copies":2864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15945,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'18963","reported_seq":27312,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:57.002917+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:23:57.002917+0000","last_peered":"2024-09-11T01:23:57.002917+0000","last_clean":"2024-09-11T01:23:57.002917+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:23:57.002917+0000","last_undegraded":"2024-09-11T01:23:57.002917+0000","last_fullsized":"2024-09-11T01:23:57.002917+0000","mapping_epoch":40,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267939,"num_objects":1287,"num_object_clones":0,"num_object_copies":2574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1287,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16830,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19157","reported_seq":19442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.700998+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:53.700998+0000","last_peered":"2024-09-11T01:23:53.700998+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:53.700998+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5885952,"num_objects":1437,"num_object_clones":0,"num_object_copies":2874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15521,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2593}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18753","reported_seq":26916,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.213512+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:23:56.213512+0000","last_peered":"2024-09-11T01:23:56.213512+0000","last_clean":"2024-09-11T01:23:56.213512+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:23:56.213512+0000","last_undegraded":"2024-09-11T01:23:56.213512+0000","last_fullsized":"2024-09-11T01:23:56.213512+0000","mapping_epoch":44,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677539,"num_objects":1387,"num_object_clones":0,"num_object_copies":2774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1387,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16522,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19235","reported_seq":19394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:56.172865+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:23:56.172865+0000","last_peered":"2024-09-11T01:23:56.172865+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:23:56.172865+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5435392,"num_objects":1327,"num_object_clones":0,"num_object_copies":2654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2820}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19069","reported_seq":27544,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:56.972373+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:23:56.972373+0000","last_peered":"2024-09-11T01:23:56.972373+0000","last_clean":"2024-09-11T01:23:56.972373+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:23:56.972373+0000","last_undegraded":"2024-09-11T01:23:56.972373+0000","last_fullsized":"2024-09-11T01:23:56.972373+0000","mapping_epoch":42,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5255168,"num_objects":1283,"num_object_clones":0,"num_object_copies":2566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15759,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19287","reported_seq":19576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.551182+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:53.551182+0000","last_peered":"2024-09-11T01:23:53.551182+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:53.551182+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5566464,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15680,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19143","reported_seq":19448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.530793+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:53.530793+0000","last_peered":"2024-09-11T01:23:53.530793+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:53.530793+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5967872,"num_objects":1457,"num_object_clones":0,"num_object_copies":2914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15678,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843102+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843102+0000","last_peered":"2024-09-11T01:23:46.843102+0000","last_clean":"2024-09-11T01:23:46.843102+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843102+0000","last_undegraded":"2024-09-11T01:23:46.843102+0000","last_fullsized":"2024-09-11T01:23:46.843102+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18761","reported_seq":19006,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:53.598662+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:53.598662+0000","last_peered":"2024-09-11T01:23:53.598662+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:53.598662+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8700","ondisk_log_start":"41'8700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5641158,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15418,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2723},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":90555126,"num_objects":22118,"num_object_clones":0,"num_object_copies":44236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22118,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":263123,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13101,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381779968,"data_stored":381627304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160960,"ondisk_log_size":160960,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493328,"kb_used_data":177024,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93878512,"statfs":{"total":96636764160,"available":96131596288,"internally_reserved":0,"allocated":181272576,"data_stored":180980247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":323874478},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790056,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537388,"kb_used_data":64220,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93834452,"statfs":{"total":96636764160,"available":96086478848,"internally_reserved":0,"allocated":65761280,"data_stored":65428744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571908,"kb_used_data":134900,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93799932,"statfs":{"total":96636764160,"available":96051130368,"internally_reserved":0,"allocated":138137600,"data_stored":137845266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":447475370},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":136654848,"data_stored":136618718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64417792,"data_stored":64341460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":180707328,"data_stored":180667126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:03.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:03.804 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:03.804 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:03.829 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-11T01:24:02.638913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80925449,"num_objects":19769,"num_object_clones":0,"num_object_copies":39538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19769,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":265476,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13178,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1563084,"kb_used_data":357396,"kb_used_omap":12,"kb_used_meta":1205619,"kb_avail":281552436,"statfs":{"total":289910292480,"available":288309694464,"internally_reserved":0,"allocated":365973504,"data_stored":365056305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1234553863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,4,5,9,14,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22724608,"num_objects":-5548,"num_object_clones":0,"num_object_copies":-11096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5548,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005364"},"pg_stats":[{"pgid":"2.f","version":"45'18948","reported_seq":19742,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.683050+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:58.683050+0000","last_peered":"2024-09-11T01:23:58.683050+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:58.683050+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18948,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2475}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19300","reported_seq":19384,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.237025+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:01.237025+0000","last_peered":"2024-09-11T01:24:01.237025+0000","last_clean":"2024-09-11T01:24:01.237025+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:01.237025+0000","last_undegraded":"2024-09-11T01:24:01.237025+0000","last_fullsized":"2024-09-11T01:24:01.237025+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5006278,"num_objects":1224,"num_object_clones":0,"num_object_copies":2448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1224,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17398,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18871","reported_seq":19045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:01.245451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:01.245451+0000","last_peered":"2024-09-11T01:24:01.245451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:01.245451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4808728,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16932,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19155","reported_seq":19205,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.251488+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:01.251488+0000","last_peered":"2024-09-11T01:24:01.251488+0000","last_clean":"2024-09-11T01:24:01.251488+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:01.251488+0000","last_undegraded":"2024-09-11T01:24:01.251488+0000","last_fullsized":"2024-09-11T01:24:01.251488+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4943872,"num_objects":1207,"num_object_clones":0,"num_object_copies":2414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19155,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18977","reported_seq":19104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:58.673926+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:58.673926+0000","last_peered":"2024-09-11T01:23:58.673926+0000","last_clean":"2024-09-11T01:23:58.673926+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:58.673926+0000","last_undegraded":"2024-09-11T01:23:58.673926+0000","last_fullsized":"2024-09-11T01:23:58.673926+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5325283,"num_objects":1301,"num_object_clones":0,"num_object_copies":2602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15775,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18992","reported_seq":19264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.705561+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:58.705561+0000","last_peered":"2024-09-11T01:23:58.705561+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:58.705561+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4977123,"num_objects":1216,"num_object_clones":0,"num_object_copies":2432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15480,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2833},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19536","reported_seq":19828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.683665+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:58.683665+0000","last_peered":"2024-09-11T01:23:58.683665+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:58.683665+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227462,"num_objects":1278,"num_object_clones":0,"num_object_copies":2556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1278,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16289,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2928},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19133","reported_seq":19258,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:58.697544+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:58.697544+0000","last_peered":"2024-09-11T01:23:58.697544+0000","last_clean":"2024-09-11T01:23:58.697544+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:58.697544+0000","last_undegraded":"2024-09-11T01:23:58.697544+0000","last_fullsized":"2024-09-11T01:23:58.697544+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5287936,"num_objects":1291,"num_object_clones":0,"num_object_copies":2582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16086,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19097","reported_seq":27446,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.942612+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:01.942612+0000","last_peered":"2024-09-11T01:24:01.942612+0000","last_clean":"2024-09-11T01:24:01.942612+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:01.942612+0000","last_undegraded":"2024-09-11T01:24:01.942612+0000","last_fullsized":"2024-09-11T01:24:01.942612+0000","mapping_epoch":40,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719075,"num_objects":1153,"num_object_clones":0,"num_object_copies":2306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16964,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19291","reported_seq":19576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.706583+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:58.706583+0000","last_peered":"2024-09-11T01:23:58.706583+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:58.706583+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5337088,"num_objects":1303,"num_object_clones":0,"num_object_copies":2606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15655,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18895","reported_seq":27058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.208574+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:01.208574+0000","last_peered":"2024-09-11T01:24:01.208574+0000","last_clean":"2024-09-11T01:24:01.208574+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:01.208574+0000","last_undegraded":"2024-09-11T01:24:01.208574+0000","last_fullsized":"2024-09-11T01:24:01.208574+0000","mapping_epoch":44,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16664,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19383","reported_seq":19542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:01.230312+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:01.230312+0000","last_peered":"2024-09-11T01:24:01.230312+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:01.230312+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4829184,"num_objects":1179,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17020,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19216","reported_seq":27692,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.978079+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:01.978079+0000","last_peered":"2024-09-11T01:24:01.978079+0000","last_clean":"2024-09-11T01:24:01.978079+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:01.978079+0000","last_undegraded":"2024-09-11T01:24:01.978079+0000","last_fullsized":"2024-09-11T01:24:01.978079+0000","mapping_epoch":42,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":1136,"num_object_clones":0,"num_object_copies":2272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15906,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19418","reported_seq":19708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.726420+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:58.726420+0000","last_peered":"2024-09-11T01:23:58.726420+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:58.726420+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5029888,"num_objects":1228,"num_object_clones":0,"num_object_copies":2456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15811,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19307","reported_seq":19612,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.697253+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:58.697253+0000","last_peered":"2024-09-11T01:23:58.697253+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:58.697253+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5296128,"num_objects":1293,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15842,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2772}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18892","reported_seq":19137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.661759+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:58.661759+0000","last_peered":"2024-09-11T01:23:58.661759+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:58.661759+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5104582,"num_objects":1248,"num_object_clones":0,"num_object_copies":2496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15549,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2854},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":80925430,"num_objects":19767,"num_object_clones":0,"num_object_copies":39534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19767,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":265474,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13177,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":362500096,"data_stored":362347432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161011,"ondisk_log_size":161011,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492652,"kb_used_data":172188,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93879188,"statfs":{"total":96636764160,"available":96132288512,"internally_reserved":0,"allocated":176320512,"data_stored":176028183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790057,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517920,"kb_used_data":57228,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93853920,"statfs":{"total":96636764160,"available":96106414080,"internally_reserved":0,"allocated":58601472,"data_stored":58268936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552512,"kb_used_data":127980,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93819328,"statfs":{"total":96636764160,"available":96070991872,"internally_reserved":0,"allocated":131051520,"data_stored":130759186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,6,3,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129413120,"data_stored":129376990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57307136,"data_stored":57230804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175779840,"data_stored":175739638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:03.830 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:04.398 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:04.399 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:04.423 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-11T01:24:02.638913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80925449,"num_objects":19769,"num_object_clones":0,"num_object_copies":39538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19769,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":265476,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13178,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1563084,"kb_used_data":357396,"kb_used_omap":12,"kb_used_meta":1205619,"kb_avail":281552436,"statfs":{"total":289910292480,"available":288309694464,"internally_reserved":0,"allocated":365973504,"data_stored":365056305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1234553863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,4,5,9,14,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22724608,"num_objects":-5548,"num_object_clones":0,"num_object_copies":-11096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5548,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5548,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005364"},"pg_stats":[{"pgid":"2.f","version":"45'18948","reported_seq":19742,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.683050+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:23:58.683050+0000","last_peered":"2024-09-11T01:23:58.683050+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:23:58.683050+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5283840,"num_objects":1290,"num_object_clones":0,"num_object_copies":2580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18948,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2475}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19300","reported_seq":19384,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.237025+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:01.237025+0000","last_peered":"2024-09-11T01:24:01.237025+0000","last_clean":"2024-09-11T01:24:01.237025+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:01.237025+0000","last_undegraded":"2024-09-11T01:24:01.237025+0000","last_fullsized":"2024-09-11T01:24:01.237025+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5006278,"num_objects":1224,"num_object_clones":0,"num_object_copies":2448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1224,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17398,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18871","reported_seq":19045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:01.245451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:01.245451+0000","last_peered":"2024-09-11T01:24:01.245451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:01.245451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4808728,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16932,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19155","reported_seq":19205,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.251488+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:01.251488+0000","last_peered":"2024-09-11T01:24:01.251488+0000","last_clean":"2024-09-11T01:24:01.251488+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:01.251488+0000","last_undegraded":"2024-09-11T01:24:01.251488+0000","last_fullsized":"2024-09-11T01:24:01.251488+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4943872,"num_objects":1207,"num_object_clones":0,"num_object_copies":2414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19155,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'18977","reported_seq":19104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:58.673926+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:23:58.673926+0000","last_peered":"2024-09-11T01:23:58.673926+0000","last_clean":"2024-09-11T01:23:58.673926+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:23:58.673926+0000","last_undegraded":"2024-09-11T01:23:58.673926+0000","last_fullsized":"2024-09-11T01:23:58.673926+0000","mapping_epoch":31,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5325283,"num_objects":1301,"num_object_clones":0,"num_object_copies":2602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1301,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15775,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'18992","reported_seq":19264,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.705561+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:23:58.705561+0000","last_peered":"2024-09-11T01:23:58.705561+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:23:58.705561+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4977123,"num_objects":1216,"num_object_clones":0,"num_object_copies":2432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15480,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2833},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19536","reported_seq":19828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.683665+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:23:58.683665+0000","last_peered":"2024-09-11T01:23:58.683665+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:23:58.683665+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5227462,"num_objects":1278,"num_object_clones":0,"num_object_copies":2556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1278,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16289,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2928},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19133","reported_seq":19258,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:58.697544+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:23:58.697544+0000","last_peered":"2024-09-11T01:23:58.697544+0000","last_clean":"2024-09-11T01:23:58.697544+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:23:58.697544+0000","last_undegraded":"2024-09-11T01:23:58.697544+0000","last_fullsized":"2024-09-11T01:23:58.697544+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5287936,"num_objects":1291,"num_object_clones":0,"num_object_copies":2582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16086,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19097","reported_seq":27446,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.942612+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:01.942612+0000","last_peered":"2024-09-11T01:24:01.942612+0000","last_clean":"2024-09-11T01:24:01.942612+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:01.942612+0000","last_undegraded":"2024-09-11T01:24:01.942612+0000","last_fullsized":"2024-09-11T01:24:01.942612+0000","mapping_epoch":40,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719075,"num_objects":1153,"num_object_clones":0,"num_object_copies":2306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16964,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19291","reported_seq":19576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.706583+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:23:58.706583+0000","last_peered":"2024-09-11T01:23:58.706583+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:23:58.706583+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5337088,"num_objects":1303,"num_object_clones":0,"num_object_copies":2606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1303,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15655,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18895","reported_seq":27058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.208574+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:01.208574+0000","last_peered":"2024-09-11T01:24:01.208574+0000","last_clean":"2024-09-11T01:24:01.208574+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:01.208574+0000","last_undegraded":"2024-09-11T01:24:01.208574+0000","last_fullsized":"2024-09-11T01:24:01.208574+0000","mapping_epoch":44,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16664,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19383","reported_seq":19542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:01.230312+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:01.230312+0000","last_peered":"2024-09-11T01:24:01.230312+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:01.230312+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4829184,"num_objects":1179,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17020,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19216","reported_seq":27692,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.978079+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:01.978079+0000","last_peered":"2024-09-11T01:24:01.978079+0000","last_clean":"2024-09-11T01:24:01.978079+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:01.978079+0000","last_undegraded":"2024-09-11T01:24:01.978079+0000","last_fullsized":"2024-09-11T01:24:01.978079+0000","mapping_epoch":42,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":1136,"num_object_clones":0,"num_object_copies":2272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15906,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19418","reported_seq":19708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.726420+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:23:58.726420+0000","last_peered":"2024-09-11T01:23:58.726420+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:23:58.726420+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5029888,"num_objects":1228,"num_object_clones":0,"num_object_copies":2456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15811,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2889}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19307","reported_seq":19612,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.697253+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:23:58.697253+0000","last_peered":"2024-09-11T01:23:58.697253+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:23:58.697253+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5296128,"num_objects":1293,"num_object_clones":0,"num_object_copies":2586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15842,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2772}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'18892","reported_seq":19137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:58.661759+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:23:58.661759+0000","last_peered":"2024-09-11T01:23:58.661759+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:23:58.661759+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5104582,"num_objects":1248,"num_object_clones":0,"num_object_copies":2496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15549,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2854},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":80925430,"num_objects":19767,"num_object_clones":0,"num_object_copies":39534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36457,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19767,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":265474,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13177,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":362500096,"data_stored":362347432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161011,"ondisk_log_size":161011,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492652,"kb_used_data":172188,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93879188,"statfs":{"total":96636764160,"available":96132288512,"internally_reserved":0,"allocated":176320512,"data_stored":176028183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790057,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517920,"kb_used_data":57228,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93853920,"statfs":{"total":96636764160,"available":96106414080,"internally_reserved":0,"allocated":58601472,"data_stored":58268936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552512,"kb_used_data":127980,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93819328,"statfs":{"total":96636764160,"available":96070991872,"internally_reserved":0,"allocated":131051520,"data_stored":130759186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":434695850},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,6,3,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":129413120,"data_stored":129376990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57307136,"data_stored":57230804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175779840,"data_stored":175739638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:04.424 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:04.424 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'18948", 'reported_seq': 19742, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.683050+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:23:58.683050+0000', 'last_peered': '2024-09-11T01:23:58.683050+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:23:58.683050+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'8900", 'ondisk_log_start': "41'8900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5283840, 'num_objects': 1290, 'num_object_clones': 0, 'num_object_copies': 2580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3765, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18948, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 358, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2475}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.424 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:04.424 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'18871", 'reported_seq': 19045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:01.245451+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:01.245451+0000', 'last_peered': '2024-09-11T01:24:01.245451+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:01.245451+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'8800", 'ondisk_log_start': "41'8800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4808728, 'num_objects': 1175, 'num_object_clones': 0, 'num_object_copies': 2350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1175, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16932, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2788}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'18992", 'reported_seq': 19264, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.705561+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:23:58.705561+0000', 'last_peered': '2024-09-11T01:23:58.705561+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:23:58.705561+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'8900", 'ondisk_log_start': "41'8900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4977123, 'num_objects': 1216, 'num_object_clones': 0, 'num_object_copies': 2432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1216, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15480, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2833}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'19536", 'reported_seq': 19828, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.683665+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:23:58.683665+0000', 'last_peered': '2024-09-11T01:23:58.683665+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:23:58.683665+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5227462, 'num_objects': 1278, 'num_object_clones': 0, 'num_object_copies': 2556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1278, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16289, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2928}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'19291", 'reported_seq': 19576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.706583+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:23:58.706583+0000', 'last_peered': '2024-09-11T01:23:58.706583+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:23:58.706583+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'9200", 'ondisk_log_start': "41'9200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5337088, 'num_objects': 1303, 'num_object_clones': 0, 'num_object_copies': 2606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1303, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15655, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2727}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'19383", 'reported_seq': 19542, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:01.230312+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:01.230312+0000', 'last_peered': '2024-09-11T01:24:01.230312+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:01.230312+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'9300", 'ondisk_log_start': "41'9300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4829184, 'num_objects': 1179, 'num_object_clones': 0, 'num_object_copies': 2358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17020, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2968}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:04.425 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'19418", 'reported_seq': 19708, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.726420+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:23:58.726420+0000', 'last_peered': '2024-09-11T01:23:58.726420+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:23:58.726420+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9400", 'ondisk_log_start': "41'9400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5029888, 'num_objects': 1228, 'num_object_clones': 0, 'num_object_copies': 2456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15811, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2889}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.426 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:04.426 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'19307", 'reported_seq': 19612, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.697253+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:23:58.697253+0000', 'last_peered': '2024-09-11T01:23:58.697253+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:23:58.697253+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'9300", 'ondisk_log_start': "41'9300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5296128, 'num_objects': 1293, 'num_object_clones': 0, 'num_object_copies': 2586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15842, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2772}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.426 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:04.426 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'18892", 'reported_seq': 19137, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:58.661759+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:23:58.661759+0000', 'last_peered': '2024-09-11T01:23:58.661759+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:23:58.661759+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'8800", 'ondisk_log_start': "41'8800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5104582, 'num_objects': 1248, 'num_object_clones': 0, 'num_object_copies': 2496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1248, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15549, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2854}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:05.182 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:05.182 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36380/36858 objects degraded (98.703%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":645,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":18429,"data_bytes":75436809,"bytes_used":1620234240,"bytes_avail":288290058240,"bytes_total":289910292480,"degraded_objects":36380,"degraded_total":36858,"degraded_ratio":0.98703130934939498,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":495},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:05.208 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:05.209 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:05.765 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:05.768 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:05.795 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-11T01:24:04.639413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":75436809,"num_objects":18429,"num_object_clones":0,"num_object_copies":36858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18429,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":266816,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13255,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160954,"ondisk_log_size":160954,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1582260,"kb_used_data":350204,"kb_used_omap":12,"kb_used_meta":1231987,"kb_avail":281533260,"statfs":{"total":289910292480,"available":288290058240,"internally_reserved":0,"allocated":358608896,"data_stored":357691697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1261554695},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,6,2,6,20,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24342528,"num_objects":-5943,"num_object_clones":0,"num_object_copies":-11886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5943,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005374"},"pg_stats":[{"pgid":"2.f","version":"45'19091","reported_seq":20041,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.724276+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:03.724276+0000","last_peered":"2024-09-11T01:24:03.724276+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:03.724276+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4698112,"num_objects":1147,"num_object_clones":0,"num_object_copies":2294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19091,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":435,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19300","reported_seq":19384,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.237025+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:01.237025+0000","last_peered":"2024-09-11T01:24:01.237025+0000","last_clean":"2024-09-11T01:24:01.237025+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:01.237025+0000","last_undegraded":"2024-09-11T01:24:01.237025+0000","last_fullsized":"2024-09-11T01:24:01.237025+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5006278,"num_objects":1224,"num_object_clones":0,"num_object_copies":2448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1224,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17398,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18871","reported_seq":19045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:01.245451+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:01.245451+0000","last_peered":"2024-09-11T01:24:01.245451+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:01.245451+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4808728,"num_objects":1175,"num_object_clones":0,"num_object_copies":2350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16932,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19155","reported_seq":19205,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.251488+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:01.251488+0000","last_peered":"2024-09-11T01:24:01.251488+0000","last_clean":"2024-09-11T01:24:01.251488+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:01.251488+0000","last_undegraded":"2024-09-11T01:24:01.251488+0000","last_fullsized":"2024-09-11T01:24:01.251488+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4943872,"num_objects":1207,"num_object_clones":0,"num_object_copies":2414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19155,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19140","reported_seq":19267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:03.680126+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:03.680126+0000","last_peered":"2024-09-11T01:24:03.680126+0000","last_clean":"2024-09-11T01:24:03.680126+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:03.680126+0000","last_undegraded":"2024-09-11T01:24:03.680126+0000","last_fullsized":"2024-09-11T01:24:03.680126+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4657635,"num_objects":1138,"num_object_clones":0,"num_object_copies":2276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15938,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19121","reported_seq":19393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.668648+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:03.668648+0000","last_peered":"2024-09-11T01:24:03.668648+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:03.668648+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4448739,"num_objects":1087,"num_object_clones":0,"num_object_copies":2174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15609,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2962},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19704","reported_seq":19996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.715266+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:03.715266+0000","last_peered":"2024-09-11T01:24:03.715266+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:03.715266+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4539334,"num_objects":1110,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1110,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16457,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3096},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19275","reported_seq":19400,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:03.695782+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:03.695782+0000","last_peered":"2024-09-11T01:24:03.695782+0000","last_clean":"2024-09-11T01:24:03.695782+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:03.695782+0000","last_undegraded":"2024-09-11T01:24:03.695782+0000","last_fullsized":"2024-09-11T01:24:03.695782+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16228,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19097","reported_seq":27446,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.942612+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:01.942612+0000","last_peered":"2024-09-11T01:24:01.942612+0000","last_clean":"2024-09-11T01:24:01.942612+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:01.942612+0000","last_undegraded":"2024-09-11T01:24:01.942612+0000","last_fullsized":"2024-09-11T01:24:01.942612+0000","mapping_epoch":40,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4719075,"num_objects":1153,"num_object_clones":0,"num_object_copies":2306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16964,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19435","reported_seq":19720,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.698714+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:03.698714+0000","last_peered":"2024-09-11T01:24:03.698714+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:03.698714+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4747264,"num_objects":1159,"num_object_clones":0,"num_object_copies":2318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15799,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2871}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'18895","reported_seq":27058,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.208574+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:01.208574+0000","last_peered":"2024-09-11T01:24:01.208574+0000","last_clean":"2024-09-11T01:24:01.208574+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:01.208574+0000","last_undegraded":"2024-09-11T01:24:01.208574+0000","last_fullsized":"2024-09-11T01:24:01.208574+0000","mapping_epoch":44,"log_start":"41'8800","ondisk_log_start":"41'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5095907,"num_objects":1245,"num_object_clones":0,"num_object_copies":2490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1245,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16664,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19383","reported_seq":19542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:01.230312+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:01.230312+0000","last_peered":"2024-09-11T01:24:01.230312+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:01.230312+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4829184,"num_objects":1179,"num_object_clones":0,"num_object_copies":2358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17020,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2968}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19216","reported_seq":27692,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:01.978079+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:01.978079+0000","last_peered":"2024-09-11T01:24:01.978079+0000","last_clean":"2024-09-11T01:24:01.978079+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:01.978079+0000","last_undegraded":"2024-09-11T01:24:01.978079+0000","last_fullsized":"2024-09-11T01:24:01.978079+0000","mapping_epoch":42,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4653056,"num_objects":1136,"num_object_clones":0,"num_object_copies":2272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15906,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19569","reported_seq":19858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.664359+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:03.664359+0000","last_peered":"2024-09-11T01:24:03.664359+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:03.664359+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4411392,"num_objects":1077,"num_object_clones":0,"num_object_copies":2154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15962,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3040}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19451","reported_seq":19756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.664741+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:03.664741+0000","last_peered":"2024-09-11T01:24:03.664741+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:03.664741+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15986,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19048","reported_seq":19293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.665102+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:03.665102+0000","last_peered":"2024-09-11T01:24:03.665102+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:03.665102+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4465606,"num_objects":1092,"num_object_clones":0,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15705,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3010},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":75436790,"num_objects":18427,"num_object_clones":0,"num_object_copies":36854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18427,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":266814,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13254,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":355201024,"data_stored":355048360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492652,"kb_used_data":172188,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93879188,"statfs":{"total":96636764160,"available":96132288512,"internally_reserved":0,"allocated":176320512,"data_stored":176028183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":328134318},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790057,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517920,"kb_used_data":57228,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93853920,"statfs":{"total":96636764160,"available":96106414080,"internally_reserved":0,"allocated":58601472,"data_stored":58268936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571688,"kb_used_data":120788,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93800152,"statfs":{"total":96636764160,"available":96051355648,"internally_reserved":0,"allocated":123686912,"data_stored":123394578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,3,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122114048,"data_stored":122077918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57307136,"data_stored":57230804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":175779840,"data_stored":175739638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:08.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:09.406 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:09.409 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:09.429 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-11T01:24:08.641061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":71406345,"num_objects":17445,"num_object_clones":0,"num_object_copies":34890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17445,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":267800,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13255,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1601400,"kb_used_data":338820,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281514120,"statfs":{"total":289910292480,"available":288270458880,"internally_reserved":0,"allocated":346951680,"data_stored":346034481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1292815367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,7,15,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20320256,"num_objects":-4961,"num_object_clones":0,"num_object_copies":-9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4961,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"45'19091","reported_seq":20041,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.724276+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:03.724276+0000","last_peered":"2024-09-11T01:24:03.724276+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:03.724276+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4698112,"num_objects":1147,"num_object_clones":0,"num_object_copies":2294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19091,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":435,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19436","reported_seq":19522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.165721+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:06.165721+0000","last_peered":"2024-09-11T01:24:06.165721+0000","last_clean":"2024-09-11T01:24:06.165721+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:06.165721+0000","last_undegraded":"2024-09-11T01:24:06.165721+0000","last_fullsized":"2024-09-11T01:24:06.165721+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4449222,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17534,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18997","reported_seq":19171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:06.259222+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:06.259222+0000","last_peered":"2024-09-11T01:24:06.259222+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:06.259222+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4292632,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17058,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2914}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19305","reported_seq":19356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.210562+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:06.210562+0000","last_peered":"2024-09-11T01:24:06.210562+0000","last_clean":"2024-09-11T01:24:06.210562+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:06.210562+0000","last_undegraded":"2024-09-11T01:24:06.210562+0000","last_fullsized":"2024-09-11T01:24:06.210562+0000","mapping_epoch":31,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4329472,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19140","reported_seq":19267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:03.680126+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:03.680126+0000","last_peered":"2024-09-11T01:24:03.680126+0000","last_clean":"2024-09-11T01:24:03.680126+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:03.680126+0000","last_undegraded":"2024-09-11T01:24:03.680126+0000","last_fullsized":"2024-09-11T01:24:03.680126+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4657635,"num_objects":1138,"num_object_clones":0,"num_object_copies":2276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15938,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19121","reported_seq":19393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.668648+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:03.668648+0000","last_peered":"2024-09-11T01:24:03.668648+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:03.668648+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4448739,"num_objects":1087,"num_object_clones":0,"num_object_copies":2174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15609,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2962},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19704","reported_seq":19996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.715266+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:03.715266+0000","last_peered":"2024-09-11T01:24:03.715266+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:03.715266+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4539334,"num_objects":1110,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1110,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16457,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3096},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19275","reported_seq":19400,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:03.695782+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:03.695782+0000","last_peered":"2024-09-11T01:24:03.695782+0000","last_clean":"2024-09-11T01:24:03.695782+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:03.695782+0000","last_undegraded":"2024-09-11T01:24:03.695782+0000","last_fullsized":"2024-09-11T01:24:03.695782+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16228,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19236","reported_seq":27585,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:07.036416+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:07.036416+0000","last_peered":"2024-09-11T01:24:07.036416+0000","last_clean":"2024-09-11T01:24:07.036416+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:07.036416+0000","last_undegraded":"2024-09-11T01:24:07.036416+0000","last_fullsized":"2024-09-11T01:24:07.036416+0000","mapping_epoch":40,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4149731,"num_objects":1014,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17103,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19435","reported_seq":19720,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.698714+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:03.698714+0000","last_peered":"2024-09-11T01:24:03.698714+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:03.698714+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4747264,"num_objects":1159,"num_object_clones":0,"num_object_copies":2318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15799,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2871}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19037","reported_seq":27200,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.224665+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:06.224665+0000","last_peered":"2024-09-11T01:24:06.224665+0000","last_clean":"2024-09-11T01:24:06.224665+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:06.224665+0000","last_undegraded":"2024-09-11T01:24:06.224665+0000","last_fullsized":"2024-09-11T01:24:06.224665+0000","mapping_epoch":44,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4514275,"num_objects":1103,"num_object_clones":0,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16806,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19542","reported_seq":19700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:06.228434+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:06.228434+0000","last_peered":"2024-09-11T01:24:06.228434+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:06.228434+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4177920,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17179,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19348","reported_seq":27823,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:07.072381+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:07.072381+0000","last_peered":"2024-09-11T01:24:07.072381+0000","last_clean":"2024-09-11T01:24:07.072381+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:07.072381+0000","last_undegraded":"2024-09-11T01:24:07.072381+0000","last_fullsized":"2024-09-11T01:24:07.072381+0000","mapping_epoch":42,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19569","reported_seq":19858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.664359+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:03.664359+0000","last_peered":"2024-09-11T01:24:03.664359+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:03.664359+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4411392,"num_objects":1077,"num_object_clones":0,"num_object_copies":2154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15962,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3040}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19451","reported_seq":19756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.664741+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:03.664741+0000","last_peered":"2024-09-11T01:24:03.664741+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:03.664741+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15986,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19048","reported_seq":19293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.665102+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:03.665102+0000","last_peered":"2024-09-11T01:24:03.665102+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:03.665102+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4465606,"num_objects":1092,"num_object_clones":0,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15705,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3010},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":71406326,"num_objects":17443,"num_object_clones":0,"num_object_copies":34886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17443,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":267798,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13254,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":343511040,"data_stored":343358376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510208,"kb_used_data":167536,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93861632,"statfs":{"total":96636764160,"available":96114311168,"internally_reserved":0,"allocated":171556864,"data_stored":171264535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790058,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519504,"kb_used_data":50496,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93852336,"statfs":{"total":96636764160,"available":96104792064,"internally_reserved":0,"allocated":51707904,"data_stored":51375368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571688,"kb_used_data":120788,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93800152,"statfs":{"total":96636764160,"available":96051355648,"internally_reserved":0,"allocated":123686912,"data_stored":123394578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,3,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122114048,"data_stored":122077918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50266112,"data_stored":50189780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171130880,"data_stored":171090678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:09.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:10.019 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:10.020 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:10.046 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-11T01:24:08.641061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":71406345,"num_objects":17445,"num_object_clones":0,"num_object_copies":34890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17445,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":267800,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13255,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1601400,"kb_used_data":338820,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281514120,"statfs":{"total":289910292480,"available":288270458880,"internally_reserved":0,"allocated":346951680,"data_stored":346034481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1292815367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,7,15,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20320256,"num_objects":-4961,"num_object_clones":0,"num_object_copies":-9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4961,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"45'19091","reported_seq":20041,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.724276+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:03.724276+0000","last_peered":"2024-09-11T01:24:03.724276+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:03.724276+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4698112,"num_objects":1147,"num_object_clones":0,"num_object_copies":2294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3688,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19091,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":435,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2541}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19436","reported_seq":19522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.165721+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:06.165721+0000","last_peered":"2024-09-11T01:24:06.165721+0000","last_clean":"2024-09-11T01:24:06.165721+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:06.165721+0000","last_undegraded":"2024-09-11T01:24:06.165721+0000","last_fullsized":"2024-09-11T01:24:06.165721+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4449222,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17534,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18997","reported_seq":19171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:06.259222+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:06.259222+0000","last_peered":"2024-09-11T01:24:06.259222+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:06.259222+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4292632,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17058,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2914}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19305","reported_seq":19356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.210562+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:06.210562+0000","last_peered":"2024-09-11T01:24:06.210562+0000","last_clean":"2024-09-11T01:24:06.210562+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:06.210562+0000","last_undegraded":"2024-09-11T01:24:06.210562+0000","last_fullsized":"2024-09-11T01:24:06.210562+0000","mapping_epoch":31,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4329472,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19140","reported_seq":19267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:03.680126+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:03.680126+0000","last_peered":"2024-09-11T01:24:03.680126+0000","last_clean":"2024-09-11T01:24:03.680126+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:03.680126+0000","last_undegraded":"2024-09-11T01:24:03.680126+0000","last_fullsized":"2024-09-11T01:24:03.680126+0000","mapping_epoch":31,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4657635,"num_objects":1138,"num_object_clones":0,"num_object_copies":2276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15938,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19121","reported_seq":19393,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.668648+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:03.668648+0000","last_peered":"2024-09-11T01:24:03.668648+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:03.668648+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4448739,"num_objects":1087,"num_object_clones":0,"num_object_copies":2174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15609,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2962},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19704","reported_seq":19996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.715266+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:03.715266+0000","last_peered":"2024-09-11T01:24:03.715266+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:03.715266+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4539334,"num_objects":1110,"num_object_clones":0,"num_object_copies":2220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1110,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16457,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3096},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19275","reported_seq":19400,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:03.695782+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:03.695782+0000","last_peered":"2024-09-11T01:24:03.695782+0000","last_clean":"2024-09-11T01:24:03.695782+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:03.695782+0000","last_undegraded":"2024-09-11T01:24:03.695782+0000","last_fullsized":"2024-09-11T01:24:03.695782+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16228,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19236","reported_seq":27585,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:07.036416+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:07.036416+0000","last_peered":"2024-09-11T01:24:07.036416+0000","last_clean":"2024-09-11T01:24:07.036416+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:07.036416+0000","last_undegraded":"2024-09-11T01:24:07.036416+0000","last_fullsized":"2024-09-11T01:24:07.036416+0000","mapping_epoch":40,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4149731,"num_objects":1014,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17103,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19435","reported_seq":19720,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.698714+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:03.698714+0000","last_peered":"2024-09-11T01:24:03.698714+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:03.698714+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4747264,"num_objects":1159,"num_object_clones":0,"num_object_copies":2318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15799,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2871}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19037","reported_seq":27200,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.224665+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:06.224665+0000","last_peered":"2024-09-11T01:24:06.224665+0000","last_clean":"2024-09-11T01:24:06.224665+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:06.224665+0000","last_undegraded":"2024-09-11T01:24:06.224665+0000","last_fullsized":"2024-09-11T01:24:06.224665+0000","mapping_epoch":44,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4514275,"num_objects":1103,"num_object_clones":0,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16806,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19542","reported_seq":19700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:06.228434+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:06.228434+0000","last_peered":"2024-09-11T01:24:06.228434+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:06.228434+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4177920,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17179,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19348","reported_seq":27823,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:07.072381+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:07.072381+0000","last_peered":"2024-09-11T01:24:07.072381+0000","last_clean":"2024-09-11T01:24:07.072381+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:07.072381+0000","last_undegraded":"2024-09-11T01:24:07.072381+0000","last_fullsized":"2024-09-11T01:24:07.072381+0000","mapping_epoch":42,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19569","reported_seq":19858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.664359+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:03.664359+0000","last_peered":"2024-09-11T01:24:03.664359+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:03.664359+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4411392,"num_objects":1077,"num_object_clones":0,"num_object_copies":2154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15962,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3040}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19451","reported_seq":19756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.664741+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:03.664741+0000","last_peered":"2024-09-11T01:24:03.664741+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:03.664741+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4706304,"num_objects":1149,"num_object_clones":0,"num_object_copies":2298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15986,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19048","reported_seq":19293,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:03.665102+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:03.665102+0000","last_peered":"2024-09-11T01:24:03.665102+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:03.665102+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4465606,"num_objects":1092,"num_object_clones":0,"num_object_copies":2184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15705,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3010},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":71406326,"num_objects":17443,"num_object_clones":0,"num_object_copies":34886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17443,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":267798,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13254,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":343511040,"data_stored":343358376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510208,"kb_used_data":167536,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93861632,"statfs":{"total":96636764160,"available":96114311168,"internally_reserved":0,"allocated":171556864,"data_stored":171264535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790058,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519504,"kb_used_data":50496,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93852336,"statfs":{"total":96636764160,"available":96104792064,"internally_reserved":0,"allocated":51707904,"data_stored":51375368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571688,"kb_used_data":120788,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93800152,"statfs":{"total":96636764160,"available":96051355648,"internally_reserved":0,"allocated":123686912,"data_stored":123394578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,2,0,3,9,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":122114048,"data_stored":122077918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50266112,"data_stored":50189780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171130880,"data_stored":171090678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'19091", 'reported_seq': 20041, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.724276+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:03.724276+0000', 'last_peered': '2024-09-11T01:24:03.724276+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:03.724276+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'9000", 'ondisk_log_start': "41'9000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4698112, 'num_objects': 1147, 'num_object_clones': 0, 'num_object_copies': 2294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3688, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19091, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 435, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2541}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'18997", 'reported_seq': 19171, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:06.259222+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:06.259222+0000', 'last_peered': '2024-09-11T01:24:06.259222+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:06.259222+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'8900", 'ondisk_log_start': "41'8900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4292632, 'num_objects': 1049, 'num_object_clones': 0, 'num_object_copies': 2098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1049, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17058, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2914}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'19121", 'reported_seq': 19393, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.668648+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:03.668648+0000', 'last_peered': '2024-09-11T01:24:03.668648+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:03.668648+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4448739, 'num_objects': 1087, 'num_object_clones': 0, 'num_object_copies': 2174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1087, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15609, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2962}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.047 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'19704", 'reported_seq': 19996, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.715266+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:03.715266+0000', 'last_peered': '2024-09-11T01:24:03.715266+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:03.715266+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'9700", 'ondisk_log_start': "41'9700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4539334, 'num_objects': 1110, 'num_object_clones': 0, 'num_object_copies': 2220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1110, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16457, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3096}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'19435", 'reported_seq': 19720, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.698714+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:03.698714+0000', 'last_peered': '2024-09-11T01:24:03.698714+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:03.698714+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'9400", 'ondisk_log_start': "41'9400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4747264, 'num_objects': 1159, 'num_object_clones': 0, 'num_object_copies': 2318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15799, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2871}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'19542", 'reported_seq': 19700, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:06.228434+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:06.228434+0000', 'last_peered': '2024-09-11T01:24:06.228434+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:06.228434+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4177920, 'num_objects': 1020, 'num_object_clones': 0, 'num_object_copies': 2040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17179, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3127}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'19569", 'reported_seq': 19858, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.664359+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:03.664359+0000', 'last_peered': '2024-09-11T01:24:03.664359+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:03.664359+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4411392, 'num_objects': 1077, 'num_object_clones': 0, 'num_object_copies': 2154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15962, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3040}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'19451", 'reported_seq': 19756, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.664741+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:03.664741+0000', 'last_peered': '2024-09-11T01:24:03.664741+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:03.664741+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'9400", 'ondisk_log_start': "41'9400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4706304, 'num_objects': 1149, 'num_object_clones': 0, 'num_object_copies': 2298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15986, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:10.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'19048", 'reported_seq': 19293, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:03.665102+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:03.665102+0000', 'last_peered': '2024-09-11T01:24:03.665102+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:03.665102+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'9000", 'ondisk_log_start': "41'9000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4465606, 'num_objects': 1092, 'num_object_clones': 0, 'num_object_copies': 2184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1092, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15705, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3010}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:10.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:10.803 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:10.804 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36380/34890 objects degraded (104.271%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":651,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":17445,"data_bytes":71406345,"bytes_used":1639833600,"bytes_avail":288270458880,"bytes_total":289910292480,"degraded_objects":36380,"degraded_total":34890,"degraded_ratio":1.0427056463169964,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":413},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:10.825 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:10.825 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:11.415 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:11.418 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:11.440 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-11T01:24:10.641877+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":65975049,"num_objects":16119,"num_object_clones":0,"num_object_copies":32238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16119,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":269126,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13341,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160764,"ondisk_log_size":160764,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1598708,"kb_used_data":331968,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281516812,"statfs":{"total":289910292480,"available":288273215488,"internally_reserved":0,"allocated":339935232,"data_stored":339018033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1297075207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,8,10,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24580096,"num_objects":-6001,"num_object_clones":0,"num_object_copies":-12002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6001,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003866"},"pg_stats":[{"pgid":"2.f","version":"45'19238","reported_seq":20360,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.773094+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:08.773094+0000","last_peered":"2024-09-11T01:24:08.773094+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:08.773094+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096000,"num_objects":1000,"num_object_clones":0,"num_object_copies":2000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2602}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19436","reported_seq":19522,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.165721+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:06.165721+0000","last_peered":"2024-09-11T01:24:06.165721+0000","last_clean":"2024-09-11T01:24:06.165721+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:06.165721+0000","last_undegraded":"2024-09-11T01:24:06.165721+0000","last_fullsized":"2024-09-11T01:24:06.165721+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4449222,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17534,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'18997","reported_seq":19171,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:06.259222+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:06.259222+0000","last_peered":"2024-09-11T01:24:06.259222+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:06.259222+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'8900","ondisk_log_start":"41'8900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4292632,"num_objects":1049,"num_object_clones":0,"num_object_copies":2098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17058,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2914}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19305","reported_seq":19356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.210562+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:06.210562+0000","last_peered":"2024-09-11T01:24:06.210562+0000","last_clean":"2024-09-11T01:24:06.210562+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:06.210562+0000","last_undegraded":"2024-09-11T01:24:06.210562+0000","last_fullsized":"2024-09-11T01:24:06.210562+0000","mapping_epoch":31,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4329472,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19305,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19294","reported_seq":19421,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:08.753255+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:08.753255+0000","last_peered":"2024-09-11T01:24:08.753255+0000","last_clean":"2024-09-11T01:24:08.753255+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:08.753255+0000","last_undegraded":"2024-09-11T01:24:08.753255+0000","last_fullsized":"2024-09-11T01:24:08.753255+0000","mapping_epoch":31,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4026851,"num_objects":984,"num_object_clones":0,"num_object_copies":1968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":984,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16092,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19265","reported_seq":19537,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.734762+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:08.734762+0000","last_peered":"2024-09-11T01:24:08.734762+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:08.734762+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3858915,"num_objects":943,"num_object_clones":0,"num_object_copies":1886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15753,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3106},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19840","reported_seq":20133,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.651587+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:08.651587+0000","last_peered":"2024-09-11T01:24:08.651587+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:08.651587+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3982278,"num_objects":974,"num_object_clones":0,"num_object_copies":1948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":974,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16593,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3232},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19430","reported_seq":19555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:08.714262+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:08.714262+0000","last_peered":"2024-09-11T01:24:08.714262+0000","last_clean":"2024-09-11T01:24:08.714262+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:08.714262+0000","last_undegraded":"2024-09-11T01:24:08.714262+0000","last_fullsized":"2024-09-11T01:24:08.714262+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071424,"num_objects":994,"num_object_clones":0,"num_object_copies":1988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16383,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19236","reported_seq":27585,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:07.036416+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:07.036416+0000","last_peered":"2024-09-11T01:24:07.036416+0000","last_clean":"2024-09-11T01:24:07.036416+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:07.036416+0000","last_undegraded":"2024-09-11T01:24:07.036416+0000","last_fullsized":"2024-09-11T01:24:07.036416+0000","mapping_epoch":40,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4149731,"num_objects":1014,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17103,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19602","reported_seq":19887,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.767489+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:08.767489+0000","last_peered":"2024-09-11T01:24:08.767489+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:08.767489+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063232,"num_objects":992,"num_object_clones":0,"num_object_copies":1984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15966,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3038}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19037","reported_seq":27200,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:06.224665+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:06.224665+0000","last_peered":"2024-09-11T01:24:06.224665+0000","last_clean":"2024-09-11T01:24:06.224665+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:06.224665+0000","last_undegraded":"2024-09-11T01:24:06.224665+0000","last_fullsized":"2024-09-11T01:24:06.224665+0000","mapping_epoch":44,"log_start":"41'9000","ondisk_log_start":"41'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4514275,"num_objects":1103,"num_object_clones":0,"num_object_copies":2206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1103,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16806,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19542","reported_seq":19700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:06.228434+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:06.228434+0000","last_peered":"2024-09-11T01:24:06.228434+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:06.228434+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4177920,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17179,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3127}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19348","reported_seq":27823,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:07.072381+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:07.072381+0000","last_peered":"2024-09-11T01:24:07.072381+0000","last_clean":"2024-09-11T01:24:07.072381+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:07.072381+0000","last_undegraded":"2024-09-11T01:24:07.072381+0000","last_fullsized":"2024-09-11T01:24:07.072381+0000","mapping_epoch":42,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16038,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19709","reported_seq":19999,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.729805+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:08.729805+0000","last_peered":"2024-09-11T01:24:08.729805+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:08.729805+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3837952,"num_objects":937,"num_object_clones":0,"num_object_copies":1874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19597","reported_seq":19902,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.779782+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:08.779782+0000","last_peered":"2024-09-11T01:24:08.779782+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:08.779782+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4108288,"num_objects":1003,"num_object_clones":0,"num_object_copies":2006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16132,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3062}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19185","reported_seq":19430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:08.705792+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:08.705792+0000","last_peered":"2024-09-11T01:24:08.705792+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:08.705792+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3904454,"num_objects":955,"num_object_clones":0,"num_object_copies":1910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15842,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":65975030,"num_objects":16117,"num_object_clones":0,"num_object_copies":32234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16117,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":269124,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13340,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":336261120,"data_stored":336108456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510208,"kb_used_data":167536,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93861632,"statfs":{"total":96636764160,"available":96114311168,"internally_reserved":0,"allocated":171556864,"data_stored":171264535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790058,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519504,"kb_used_data":50496,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93852336,"statfs":{"total":96636764160,"available":96104792064,"internally_reserved":0,"allocated":51707904,"data_stored":51375368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568996,"kb_used_data":113936,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93802844,"statfs":{"total":96636764160,"available":96054112256,"internally_reserved":0,"allocated":116670464,"data_stored":116378130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,4,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":114864128,"data_stored":114827998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50266112,"data_stored":50189780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":171130880,"data_stored":171090678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:14.442 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:15.038 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:15.039 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:15.064 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-11T01:24:14.643678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":56525577,"num_objects":13812,"num_object_clones":0,"num_object_copies":27624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13812,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":271433,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13420,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1577040,"kb_used_data":314456,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281538480,"statfs":{"total":289910292480,"available":288295403520,"internally_reserved":0,"allocated":322002944,"data_stored":321085745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1292815367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,6,6,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24399872,"num_objects":-5957,"num_object_clones":0,"num_object_copies":-11914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004765"},"pg_stats":[{"pgid":"2.f","version":"45'19380","reported_seq":20660,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.838688+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:13.838688+0000","last_peered":"2024-09-11T01:24:13.838688+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:13.838688+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19380,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2665}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19575","reported_seq":19660,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:11.194272+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:11.194272+0000","last_peered":"2024-09-11T01:24:11.194272+0000","last_clean":"2024-09-11T01:24:11.194272+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:11.194272+0000","last_undegraded":"2024-09-11T01:24:11.194272+0000","last_fullsized":"2024-09-11T01:24:11.194272+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3879878,"num_objects":949,"num_object_clones":0,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17673,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19142","reported_seq":19315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:11.303115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:11.303115+0000","last_peered":"2024-09-11T01:24:11.303115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:11.303115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3698712,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17203,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19463","reported_seq":19514,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:11.210660+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:11.210660+0000","last_peered":"2024-09-11T01:24:11.210660+0000","last_clean":"2024-09-11T01:24:11.210660+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:11.210660+0000","last_undegraded":"2024-09-11T01:24:11.210660+0000","last_fullsized":"2024-09-11T01:24:11.210660+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682304,"num_objects":899,"num_object_clones":0,"num_object_copies":1798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19463,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19430","reported_seq":19557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.764214+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:13.764214+0000","last_peered":"2024-09-11T01:24:13.764214+0000","last_clean":"2024-09-11T01:24:13.764214+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:13.764214+0000","last_undegraded":"2024-09-11T01:24:13.764214+0000","last_fullsized":"2024-09-11T01:24:13.764214+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469795,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16228,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19409","reported_seq":19681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.810043+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:13.810043+0000","last_peered":"2024-09-11T01:24:13.810043+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:13.810043+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3269091,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15897,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19989","reported_seq":20282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.736327+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:13.736327+0000","last_peered":"2024-09-11T01:24:13.736327+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:13.736327+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371974,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16742,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3381},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19577","reported_seq":19702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.753531+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:13.753531+0000","last_peered":"2024-09-11T01:24:13.753531+0000","last_clean":"2024-09-11T01:24:13.753531+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:13.753531+0000","last_undegraded":"2024-09-11T01:24:13.753531+0000","last_fullsized":"2024-09-11T01:24:13.753531+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469312,"num_objects":847,"num_object_clones":0,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19372","reported_seq":27721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:12.082465+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:12.082465+0000","last_peered":"2024-09-11T01:24:12.082465+0000","last_clean":"2024-09-11T01:24:12.082465+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:12.082465+0000","last_undegraded":"2024-09-11T01:24:12.082465+0000","last_fullsized":"2024-09-11T01:24:12.082465+0000","mapping_epoch":40,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3592675,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17239,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19731","reported_seq":20016,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.729390+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:13.729390+0000","last_peered":"2024-09-11T01:24:13.729390+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:13.729390+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3534848,"num_objects":863,"num_object_clones":0,"num_object_copies":1726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19190","reported_seq":27353,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:11.277085+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:11.277085+0000","last_peered":"2024-09-11T01:24:11.277085+0000","last_clean":"2024-09-11T01:24:11.277085+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:11.277085+0000","last_undegraded":"2024-09-11T01:24:11.277085+0000","last_fullsized":"2024-09-11T01:24:11.277085+0000","mapping_epoch":44,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3887587,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16959,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19677","reported_seq":19836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:11.230652+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:11.230652+0000","last_peered":"2024-09-11T01:24:11.230652+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:11.230652+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17314,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19504","reported_seq":27980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:12.148210+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:12.148210+0000","last_peered":"2024-09-11T01:24:12.148210+0000","last_clean":"2024-09-11T01:24:12.148210+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:12.148210+0000","last_undegraded":"2024-09-11T01:24:12.148210+0000","last_fullsized":"2024-09-11T01:24:12.148210+0000","mapping_epoch":42,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19850","reported_seq":20140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.818539+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:13.818539+0000","last_peered":"2024-09-11T01:24:13.818539+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:13.818539+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260416,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16243,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19756","reported_seq":20061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.823125+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:13.823125+0000","last_peered":"2024-09-11T01:24:13.823125+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:13.823125+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3457024,"num_objects":844,"num_object_clones":0,"num_object_copies":1688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19323","reported_seq":19568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.653591+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:13.653591+0000","last_peered":"2024-09-11T01:24:13.653591+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:13.653591+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3339206,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15980,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3285},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":56525558,"num_objects":13810,"num_object_clones":0,"num_object_copies":27620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13810,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":271431,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13419,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":317116416,"data_stored":316963752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510568,"kb_used_data":163736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93861272,"statfs":{"total":96636764160,"available":96113942528,"internally_reserved":0,"allocated":167665664,"data_stored":167373335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790059,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516904,"kb_used_data":43736,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93854936,"statfs":{"total":96636764160,"available":96107454464,"internally_reserved":0,"allocated":44785664,"data_stored":44453128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549568,"kb_used_data":106984,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93822272,"statfs":{"total":96636764160,"available":96074006528,"internally_reserved":0,"allocated":109551616,"data_stored":109259282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107655168,"data_stored":107619038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43139072,"data_stored":43062740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166322176,"data_stored":166281974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:15.066 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:15.621 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:15.621 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:15.653 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-11T01:24:14.643678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":56525577,"num_objects":13812,"num_object_clones":0,"num_object_copies":27624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13812,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":271433,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13420,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160871,"ondisk_log_size":160871,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1577040,"kb_used_data":314456,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281538480,"statfs":{"total":289910292480,"available":288295403520,"internally_reserved":0,"allocated":322002944,"data_stored":321085745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1292815367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,6,6,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24399872,"num_objects":-5957,"num_object_clones":0,"num_object_copies":-11914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5957,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004765"},"pg_stats":[{"pgid":"2.f","version":"45'19380","reported_seq":20660,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.838688+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:13.838688+0000","last_peered":"2024-09-11T01:24:13.838688+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:13.838688+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19380,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2665}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19575","reported_seq":19660,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:11.194272+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:11.194272+0000","last_peered":"2024-09-11T01:24:11.194272+0000","last_clean":"2024-09-11T01:24:11.194272+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:11.194272+0000","last_undegraded":"2024-09-11T01:24:11.194272+0000","last_fullsized":"2024-09-11T01:24:11.194272+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3879878,"num_objects":949,"num_object_clones":0,"num_object_copies":1898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":949,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17673,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19142","reported_seq":19315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:11.303115+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:11.303115+0000","last_peered":"2024-09-11T01:24:11.303115+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:11.303115+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3698712,"num_objects":904,"num_object_clones":0,"num_object_copies":1808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17203,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3059}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19463","reported_seq":19514,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:11.210660+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:11.210660+0000","last_peered":"2024-09-11T01:24:11.210660+0000","last_clean":"2024-09-11T01:24:11.210660+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:11.210660+0000","last_undegraded":"2024-09-11T01:24:11.210660+0000","last_fullsized":"2024-09-11T01:24:11.210660+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3682304,"num_objects":899,"num_object_clones":0,"num_object_copies":1798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19463,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19430","reported_seq":19557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.764214+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:13.764214+0000","last_peered":"2024-09-11T01:24:13.764214+0000","last_clean":"2024-09-11T01:24:13.764214+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:13.764214+0000","last_undegraded":"2024-09-11T01:24:13.764214+0000","last_fullsized":"2024-09-11T01:24:13.764214+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469795,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16228,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19409","reported_seq":19681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.810043+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:13.810043+0000","last_peered":"2024-09-11T01:24:13.810043+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:13.810043+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3269091,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15897,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19989","reported_seq":20282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.736327+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:13.736327+0000","last_peered":"2024-09-11T01:24:13.736327+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:13.736327+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371974,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16742,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3381},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19577","reported_seq":19702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.753531+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:13.753531+0000","last_peered":"2024-09-11T01:24:13.753531+0000","last_clean":"2024-09-11T01:24:13.753531+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:13.753531+0000","last_undegraded":"2024-09-11T01:24:13.753531+0000","last_fullsized":"2024-09-11T01:24:13.753531+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469312,"num_objects":847,"num_object_clones":0,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19372","reported_seq":27721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:12.082465+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:12.082465+0000","last_peered":"2024-09-11T01:24:12.082465+0000","last_clean":"2024-09-11T01:24:12.082465+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:12.082465+0000","last_undegraded":"2024-09-11T01:24:12.082465+0000","last_fullsized":"2024-09-11T01:24:12.082465+0000","mapping_epoch":40,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3592675,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17239,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19731","reported_seq":20016,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.729390+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:13.729390+0000","last_peered":"2024-09-11T01:24:13.729390+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:13.729390+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3534848,"num_objects":863,"num_object_clones":0,"num_object_copies":1726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19190","reported_seq":27353,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:11.277085+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:11.277085+0000","last_peered":"2024-09-11T01:24:11.277085+0000","last_clean":"2024-09-11T01:24:11.277085+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:11.277085+0000","last_undegraded":"2024-09-11T01:24:11.277085+0000","last_fullsized":"2024-09-11T01:24:11.277085+0000","mapping_epoch":44,"log_start":"41'9100","ondisk_log_start":"41'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3887587,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16959,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19677","reported_seq":19836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:11.230652+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:11.230652+0000","last_peered":"2024-09-11T01:24:11.230652+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:11.230652+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624960,"num_objects":885,"num_object_clones":0,"num_object_copies":1770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17314,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3262}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19504","reported_seq":27980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:12.148210+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:12.148210+0000","last_peered":"2024-09-11T01:24:12.148210+0000","last_clean":"2024-09-11T01:24:12.148210+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:12.148210+0000","last_undegraded":"2024-09-11T01:24:12.148210+0000","last_fullsized":"2024-09-11T01:24:12.148210+0000","mapping_epoch":42,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19850","reported_seq":20140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.818539+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:13.818539+0000","last_peered":"2024-09-11T01:24:13.818539+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:13.818539+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260416,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16243,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19756","reported_seq":20061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.823125+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:13.823125+0000","last_peered":"2024-09-11T01:24:13.823125+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:13.823125+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3457024,"num_objects":844,"num_object_clones":0,"num_object_copies":1688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19323","reported_seq":19568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.653591+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:13.653591+0000","last_peered":"2024-09-11T01:24:13.653591+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:13.653591+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3339206,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15980,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3285},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":56525558,"num_objects":13810,"num_object_clones":0,"num_object_copies":27620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13810,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":271431,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13419,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":317116416,"data_stored":316963752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510568,"kb_used_data":163736,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93861272,"statfs":{"total":96636764160,"available":96113942528,"internally_reserved":0,"allocated":167665664,"data_stored":167373335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790059,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516904,"kb_used_data":43736,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93854936,"statfs":{"total":96636764160,"available":96107454464,"internally_reserved":0,"allocated":44785664,"data_stored":44453128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549568,"kb_used_data":106984,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93822272,"statfs":{"total":96636764160,"available":96074006528,"internally_reserved":0,"allocated":109551616,"data_stored":109259282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107655168,"data_stored":107619038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43139072,"data_stored":43062740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":166322176,"data_stored":166281974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'19380", 'reported_seq': 20660, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.838688+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:13.838688+0000', 'last_peered': '2024-09-11T01:24:13.838688+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:13.838688+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'9300", 'ondisk_log_start': "41'9300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3514368, 'num_objects': 858, 'num_object_clones': 0, 'num_object_copies': 1716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3523, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19380, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 600, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2665}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'19142", 'reported_seq': 19315, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:11.303115+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:11.303115+0000', 'last_peered': '2024-09-11T01:24:11.303115+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:11.303115+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'9100", 'ondisk_log_start': "41'9100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3698712, 'num_objects': 904, 'num_object_clones': 0, 'num_object_copies': 1808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 904, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17203, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'19409", 'reported_seq': 19681, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.810043+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:13.810043+0000', 'last_peered': '2024-09-11T01:24:13.810043+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:13.810043+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'9400", 'ondisk_log_start': "41'9400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3269091, 'num_objects': 799, 'num_object_clones': 0, 'num_object_copies': 1598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 799, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15897, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3250}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:15.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'19989", 'reported_seq': 20282, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.736327+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:13.736327+0000', 'last_peered': '2024-09-11T01:24:13.736327+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:13.736327+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'9900", 'ondisk_log_start': "41'9900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3371974, 'num_objects': 825, 'num_object_clones': 0, 'num_object_copies': 1650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 825, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16742, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3381}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'19731", 'reported_seq': 20016, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.729390+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:13.729390+0000', 'last_peered': '2024-09-11T01:24:13.729390+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:13.729390+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'9700", 'ondisk_log_start': "41'9700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3534848, 'num_objects': 863, 'num_object_clones': 0, 'num_object_copies': 1726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16095, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3167}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'19677", 'reported_seq': 19836, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:11.230652+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:11.230652+0000', 'last_peered': '2024-09-11T01:24:11.230652+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:11.230652+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'9600", 'ondisk_log_start': "41'9600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3624960, 'num_objects': 885, 'num_object_clones': 0, 'num_object_copies': 1770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17314, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3262}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'19850", 'reported_seq': 20140, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.818539+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:13.818539+0000', 'last_peered': '2024-09-11T01:24:13.818539+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:13.818539+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3260416, 'num_objects': 796, 'num_object_clones': 0, 'num_object_copies': 1592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16243, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'19756", 'reported_seq': 20061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.823125+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:13.823125+0000', 'last_peered': '2024-09-11T01:24:13.823125+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:13.823125+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'9700", 'ondisk_log_start': "41'9700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3457024, 'num_objects': 844, 'num_object_clones': 0, 'num_object_copies': 1688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16291, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:15.656 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'19323", 'reported_seq': 19568, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:13.653591+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:13.653591+0000', 'last_peered': '2024-09-11T01:24:13.653591+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:13.653591+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'9300", 'ondisk_log_start': "41'9300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3339206, 'num_objects': 817, 'num_object_clones': 0, 'num_object_copies': 1634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 817, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15980, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3285}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:15.657 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:16.402 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:16.402 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36215/27624 objects degraded (131.100%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":656,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":13812,"data_bytes":56525577,"bytes_used":1614888960,"bytes_avail":288295403520,"bytes_total":289910292480,"degraded_objects":36215,"degraded_total":27624,"degraded_ratio":1.3109976831740517,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:16.428 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:16.429 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:16.917 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:16.917 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:16.942 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-11T01:24:16.644580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":53322505,"num_objects":13030,"num_object_clones":0,"num_object_copies":26060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13030,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":272215,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13420,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1559672,"kb_used_data":309568,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281555848,"statfs":{"total":289910292480,"available":288313188352,"internally_reserved":0,"allocated":316997632,"data_stored":316080433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1280035847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,7,7,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22114304,"num_objects":-5399,"num_object_clones":0,"num_object_copies":-10798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5399,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005167"},"pg_stats":[{"pgid":"2.f","version":"45'19380","reported_seq":20660,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.838688+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:13.838688+0000","last_peered":"2024-09-11T01:24:13.838688+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:13.838688+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19380,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2665}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19749","reported_seq":19835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.184151+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:16.184151+0000","last_peered":"2024-09-11T01:24:16.184151+0000","last_clean":"2024-09-11T01:24:16.184151+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:16.184151+0000","last_undegraded":"2024-09-11T01:24:16.184151+0000","last_fullsized":"2024-09-11T01:24:16.184151+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3167174,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17847,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19285","reported_seq":19455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.294993+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:16.294993+0000","last_peered":"2024-09-11T01:24:16.294993+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:16.294993+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3112984,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17346,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3202}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19606","reported_seq":19657,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.262905+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:16.262905+0000","last_peered":"2024-09-11T01:24:16.262905+0000","last_clean":"2024-09-11T01:24:16.262905+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:16.262905+0000","last_undegraded":"2024-09-11T01:24:16.262905+0000","last_fullsized":"2024-09-11T01:24:16.262905+0000","mapping_epoch":31,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3096576,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19430","reported_seq":19557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.764214+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:13.764214+0000","last_peered":"2024-09-11T01:24:13.764214+0000","last_clean":"2024-09-11T01:24:13.764214+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:13.764214+0000","last_undegraded":"2024-09-11T01:24:13.764214+0000","last_fullsized":"2024-09-11T01:24:13.764214+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469795,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16228,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19409","reported_seq":19681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.810043+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:13.810043+0000","last_peered":"2024-09-11T01:24:13.810043+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:13.810043+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3269091,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15897,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19989","reported_seq":20282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.736327+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:13.736327+0000","last_peered":"2024-09-11T01:24:13.736327+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:13.736327+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371974,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16742,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3381},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19577","reported_seq":19702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.753531+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:13.753531+0000","last_peered":"2024-09-11T01:24:13.753531+0000","last_clean":"2024-09-11T01:24:13.753531+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:13.753531+0000","last_undegraded":"2024-09-11T01:24:13.753531+0000","last_fullsized":"2024-09-11T01:24:13.753531+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469312,"num_objects":847,"num_object_clones":0,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19372","reported_seq":27721,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:12.082465+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:12.082465+0000","last_peered":"2024-09-11T01:24:12.082465+0000","last_clean":"2024-09-11T01:24:12.082465+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:12.082465+0000","last_undegraded":"2024-09-11T01:24:12.082465+0000","last_fullsized":"2024-09-11T01:24:12.082465+0000","mapping_epoch":40,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3592675,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17239,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19731","reported_seq":20016,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.729390+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:13.729390+0000","last_peered":"2024-09-11T01:24:13.729390+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:13.729390+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3534848,"num_objects":863,"num_object_clones":0,"num_object_copies":1726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19365","reported_seq":27528,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.197953+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:16.197953+0000","last_peered":"2024-09-11T01:24:16.197953+0000","last_clean":"2024-09-11T01:24:16.197953+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:16.197953+0000","last_undegraded":"2024-09-11T01:24:16.197953+0000","last_fullsized":"2024-09-11T01:24:16.197953+0000","mapping_epoch":44,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3170787,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19824","reported_seq":19983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.217857+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:16.217857+0000","last_peered":"2024-09-11T01:24:16.217857+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:16.217857+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3022848,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17461,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3409}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19504","reported_seq":27980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:12.148210+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:12.148210+0000","last_peered":"2024-09-11T01:24:12.148210+0000","last_clean":"2024-09-11T01:24:12.148210+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:12.148210+0000","last_undegraded":"2024-09-11T01:24:12.148210+0000","last_fullsized":"2024-09-11T01:24:12.148210+0000","mapping_epoch":42,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3473408,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16194,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19850","reported_seq":20140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.818539+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:13.818539+0000","last_peered":"2024-09-11T01:24:13.818539+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:13.818539+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260416,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16243,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19756","reported_seq":20061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.823125+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:13.823125+0000","last_peered":"2024-09-11T01:24:13.823125+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:13.823125+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3457024,"num_objects":844,"num_object_clones":0,"num_object_copies":1688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19323","reported_seq":19568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.653591+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:13.653591+0000","last_peered":"2024-09-11T01:24:13.653591+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:13.653591+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3339206,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15980,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3285},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":53322486,"num_objects":13028,"num_object_clones":0,"num_object_copies":26056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13028,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":272213,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13419,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":312246272,"data_stored":312093608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493200,"kb_used_data":158848,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93878640,"statfs":{"total":96636764160,"available":96131727360,"internally_reserved":0,"allocated":162660352,"data_stored":162368023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790059,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516904,"kb_used_data":43736,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93854936,"statfs":{"total":96636764160,"available":96107454464,"internally_reserved":0,"allocated":44785664,"data_stored":44453128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549568,"kb_used_data":106984,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93822272,"statfs":{"total":96636764160,"available":96074006528,"internally_reserved":0,"allocated":109551616,"data_stored":109259282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107655168,"data_stored":107619038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":43139072,"data_stored":43062740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161452032,"data_stored":161411830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:19.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:20.536 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:20.537 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:20.573 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-11T01:24:18.645498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":52159241,"num_objects":12746,"num_object_clones":0,"num_object_copies":25492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12746,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":272499,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13420,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544408,"kb_used_data":302624,"kb_used_omap":12,"kb_used_meta":1241715,"kb_avail":281571112,"statfs":{"total":289910292480,"available":288328818688,"internally_reserved":0,"allocated":309886976,"data_stored":308969777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1271516167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,8,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20357120,"num_objects":-4970,"num_object_clones":0,"num_object_copies":-9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4970,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005248"},"pg_stats":[{"pgid":"2.f","version":"45'19380","reported_seq":20660,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.838688+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:13.838688+0000","last_peered":"2024-09-11T01:24:13.838688+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:13.838688+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3514368,"num_objects":858,"num_object_clones":0,"num_object_copies":1716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19380,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2665}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19749","reported_seq":19835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.184151+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:16.184151+0000","last_peered":"2024-09-11T01:24:16.184151+0000","last_clean":"2024-09-11T01:24:16.184151+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:16.184151+0000","last_undegraded":"2024-09-11T01:24:16.184151+0000","last_fullsized":"2024-09-11T01:24:16.184151+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3167174,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17847,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19285","reported_seq":19455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.294993+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:16.294993+0000","last_peered":"2024-09-11T01:24:16.294993+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:16.294993+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3112984,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17346,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3202}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19606","reported_seq":19657,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.262905+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:16.262905+0000","last_peered":"2024-09-11T01:24:16.262905+0000","last_clean":"2024-09-11T01:24:16.262905+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:16.262905+0000","last_undegraded":"2024-09-11T01:24:16.262905+0000","last_fullsized":"2024-09-11T01:24:16.262905+0000","mapping_epoch":31,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3096576,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19430","reported_seq":19557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.764214+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:13.764214+0000","last_peered":"2024-09-11T01:24:13.764214+0000","last_clean":"2024-09-11T01:24:13.764214+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:13.764214+0000","last_undegraded":"2024-09-11T01:24:13.764214+0000","last_fullsized":"2024-09-11T01:24:13.764214+0000","mapping_epoch":31,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469795,"num_objects":848,"num_object_clones":0,"num_object_copies":1696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16228,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19409","reported_seq":19681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.810043+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:13.810043+0000","last_peered":"2024-09-11T01:24:13.810043+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:13.810043+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3269091,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15897,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3250},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'19989","reported_seq":20282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.736327+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:13.736327+0000","last_peered":"2024-09-11T01:24:13.736327+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:13.736327+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371974,"num_objects":825,"num_object_clones":0,"num_object_copies":1650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":825,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16742,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3381},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19577","reported_seq":19702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:13.753531+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:13.753531+0000","last_peered":"2024-09-11T01:24:13.753531+0000","last_clean":"2024-09-11T01:24:13.753531+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:13.753531+0000","last_undegraded":"2024-09-11T01:24:13.753531+0000","last_fullsized":"2024-09-11T01:24:13.753531+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3469312,"num_objects":847,"num_object_clones":0,"num_object_copies":1694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19507","reported_seq":27855,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:17.235758+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:17.235758+0000","last_peered":"2024-09-11T01:24:17.235758+0000","last_clean":"2024-09-11T01:24:17.235758+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:17.235758+0000","last_undegraded":"2024-09-11T01:24:17.235758+0000","last_fullsized":"2024-09-11T01:24:17.235758+0000","mapping_epoch":40,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3039715,"num_objects":743,"num_object_clones":0,"num_object_copies":1486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17374,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19731","reported_seq":20016,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.729390+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:13.729390+0000","last_peered":"2024-09-11T01:24:13.729390+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:13.729390+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3534848,"num_objects":863,"num_object_clones":0,"num_object_copies":1726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16095,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19365","reported_seq":27528,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.197953+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:16.197953+0000","last_peered":"2024-09-11T01:24:16.197953+0000","last_clean":"2024-09-11T01:24:16.197953+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:16.197953+0000","last_undegraded":"2024-09-11T01:24:16.197953+0000","last_fullsized":"2024-09-11T01:24:16.197953+0000","mapping_epoch":44,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3170787,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19824","reported_seq":19983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.217857+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:16.217857+0000","last_peered":"2024-09-11T01:24:16.217857+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:16.217857+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3022848,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17461,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3409}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19653","reported_seq":28127,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:17.218815+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:17.218815+0000","last_peered":"2024-09-11T01:24:17.218815+0000","last_clean":"2024-09-11T01:24:17.218815+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:17.218815+0000","last_undegraded":"2024-09-11T01:24:17.218815+0000","last_fullsized":"2024-09-11T01:24:17.218815+0000","mapping_epoch":42,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2863104,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19850","reported_seq":20140,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.818539+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:13.818539+0000","last_peered":"2024-09-11T01:24:13.818539+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:13.818539+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260416,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16243,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19756","reported_seq":20061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.823125+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:13.823125+0000","last_peered":"2024-09-11T01:24:13.823125+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:13.823125+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3457024,"num_objects":844,"num_object_clones":0,"num_object_copies":1688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16291,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19323","reported_seq":19568,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:13.653591+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:13.653591+0000","last_peered":"2024-09-11T01:24:13.653591+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:13.653591+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3339206,"num_objects":817,"num_object_clones":0,"num_object_copies":1634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15980,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3285},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":52159222,"num_objects":12744,"num_object_clones":0,"num_object_copies":25488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12744,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":272497,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13419,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305238016,"data_stored":305085352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493200,"kb_used_data":158848,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93878640,"statfs":{"total":96636764160,"available":96131727360,"internally_reserved":0,"allocated":162660352,"data_stored":162368023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790060,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501640,"kb_used_data":36792,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93870200,"statfs":{"total":96636764160,"available":96123084800,"internally_reserved":0,"allocated":37675008,"data_stored":37342472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549568,"kb_used_data":106984,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93822272,"statfs":{"total":96636764160,"available":96074006528,"internally_reserved":0,"allocated":109551616,"data_stored":109259282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,3,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":107655168,"data_stored":107619038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36130816,"data_stored":36054484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161452032,"data_stored":161411830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:20.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:21.187 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:21.188 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:21.215 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-11T01:24:20.646386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":46977801,"num_objects":11481,"num_object_clones":0,"num_object_copies":22962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11481,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":273764,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13503,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544536,"kb_used_data":294440,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281570984,"statfs":{"total":289910292480,"available":288328687616,"internally_reserved":0,"allocated":301506560,"data_stored":300589361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1280035847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,10,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24428544,"num_objects":-5964,"num_object_clones":0,"num_object_copies":-11928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5964,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005324"},"pg_stats":[{"pgid":"2.f","version":"45'19524","reported_seq":20969,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.785511+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:18.785511+0000","last_peered":"2024-09-11T01:24:18.785511+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:18.785511+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19524,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2726}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19749","reported_seq":19835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.184151+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:16.184151+0000","last_peered":"2024-09-11T01:24:16.184151+0000","last_clean":"2024-09-11T01:24:16.184151+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:16.184151+0000","last_undegraded":"2024-09-11T01:24:16.184151+0000","last_fullsized":"2024-09-11T01:24:16.184151+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3167174,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17847,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19285","reported_seq":19455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.294993+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:16.294993+0000","last_peered":"2024-09-11T01:24:16.294993+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:16.294993+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3112984,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17346,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3202}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19606","reported_seq":19657,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.262905+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:16.262905+0000","last_peered":"2024-09-11T01:24:16.262905+0000","last_clean":"2024-09-11T01:24:16.262905+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:16.262905+0000","last_undegraded":"2024-09-11T01:24:16.262905+0000","last_fullsized":"2024-09-11T01:24:16.262905+0000","mapping_epoch":31,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3096576,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19575","reported_seq":19702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:18.792050+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:18.792050+0000","last_peered":"2024-09-11T01:24:18.792050+0000","last_clean":"2024-09-11T01:24:18.792050+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:18.792050+0000","last_undegraded":"2024-09-11T01:24:18.792050+0000","last_fullsized":"2024-09-11T01:24:18.792050+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2875875,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16373,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19543","reported_seq":19815,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.810644+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:18.810644+0000","last_peered":"2024-09-11T01:24:18.810644+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:18.810644+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2720227,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16031,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3384},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20119","reported_seq":20412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.746616+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:18.746616+0000","last_peered":"2024-09-11T01:24:18.746616+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:18.746616+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2839494,"num_objects":695,"num_object_clones":0,"num_object_copies":1390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":695,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16872,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3511},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19714","reported_seq":19839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:18.697441+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:18.697441+0000","last_peered":"2024-09-11T01:24:18.697441+0000","last_clean":"2024-09-11T01:24:18.697441+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:18.697441+0000","last_undegraded":"2024-09-11T01:24:18.697441+0000","last_fullsized":"2024-09-11T01:24:18.697441+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2908160,"num_objects":710,"num_object_clones":0,"num_object_copies":1420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16667,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19507","reported_seq":27855,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:17.235758+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:17.235758+0000","last_peered":"2024-09-11T01:24:17.235758+0000","last_clean":"2024-09-11T01:24:17.235758+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:17.235758+0000","last_undegraded":"2024-09-11T01:24:17.235758+0000","last_fullsized":"2024-09-11T01:24:17.235758+0000","mapping_epoch":40,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3039715,"num_objects":743,"num_object_clones":0,"num_object_copies":1486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17374,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19876","reported_seq":20159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.829449+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:18.829449+0000","last_peered":"2024-09-11T01:24:18.829449+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:18.829449+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16240,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19365","reported_seq":27528,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.197953+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:16.197953+0000","last_peered":"2024-09-11T01:24:16.197953+0000","last_clean":"2024-09-11T01:24:16.197953+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:16.197953+0000","last_undegraded":"2024-09-11T01:24:16.197953+0000","last_fullsized":"2024-09-11T01:24:16.197953+0000","mapping_epoch":44,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3170787,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19824","reported_seq":19983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.217857+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:16.217857+0000","last_peered":"2024-09-11T01:24:16.217857+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:16.217857+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3022848,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17461,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3409}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19653","reported_seq":28127,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:17.218815+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:17.218815+0000","last_peered":"2024-09-11T01:24:17.218815+0000","last_clean":"2024-09-11T01:24:17.218815+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:17.218815+0000","last_undegraded":"2024-09-11T01:24:17.218815+0000","last_fullsized":"2024-09-11T01:24:17.218815+0000","mapping_epoch":42,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2863104,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19995","reported_seq":20285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.776126+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:18.776126+0000","last_peered":"2024-09-11T01:24:18.776126+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:18.776126+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2666496,"num_objects":651,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16388,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19896","reported_seq":20201,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.858739+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:18.858739+0000","last_peered":"2024-09-11T01:24:18.858739+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:18.858739+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":704,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16431,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19468","reported_seq":19713,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.816268+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:18.816268+0000","last_peered":"2024-09-11T01:24:18.816268+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:18.816268+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2745286,"num_objects":672,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16125,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3430},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":46977782,"num_objects":11479,"num_object_clones":0,"num_object_copies":22958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11479,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":273762,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13502,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298323968,"data_stored":298171304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493200,"kb_used_data":158848,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93878640,"statfs":{"total":96636764160,"available":96131727360,"internally_reserved":0,"allocated":162660352,"data_stored":162368023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790060,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501640,"kb_used_data":36792,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93870200,"statfs":{"total":96636764160,"available":96123084800,"internally_reserved":0,"allocated":37675008,"data_stored":37342472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549696,"kb_used_data":98800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93822144,"statfs":{"total":96636764160,"available":96073875456,"internally_reserved":0,"allocated":101171200,"data_stored":100878866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,7,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100741120,"data_stored":100704990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36130816,"data_stored":36054484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161452032,"data_stored":161411830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'19524", 'reported_seq': 20969, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.785511+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:18.785511+0000', 'last_peered': '2024-09-11T01:24:18.785511+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:18.785511+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2924544, 'num_objects': 714, 'num_object_clones': 0, 'num_object_copies': 1428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3440, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 714, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19524, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 683, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2726}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'19285", 'reported_seq': 19455, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:16.294993+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:16.294993+0000', 'last_peered': '2024-09-11T01:24:16.294993+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:16.294993+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'9200", 'ondisk_log_start': "41'9200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3112984, 'num_objects': 761, 'num_object_clones': 0, 'num_object_copies': 1522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 761, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17346, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3202}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'19543", 'reported_seq': 19815, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.810644+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:18.810644+0000', 'last_peered': '2024-09-11T01:24:18.810644+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:18.810644+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2720227, 'num_objects': 665, 'num_object_clones': 0, 'num_object_copies': 1330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 665, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16031, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3384}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20119", 'reported_seq': 20412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.746616+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:18.746616+0000', 'last_peered': '2024-09-11T01:24:18.746616+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:18.746616+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2839494, 'num_objects': 695, 'num_object_clones': 0, 'num_object_copies': 1390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 695, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 16872, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3511}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:21.217 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'19876", 'reported_seq': 20159, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.829449+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:18.829449+0000', 'last_peered': '2024-09-11T01:24:18.829449+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:18.829449+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2940928, 'num_objects': 718, 'num_object_clones': 0, 'num_object_copies': 1436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16240, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3312}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'19824", 'reported_seq': 19983, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:16.217857+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:16.217857+0000', 'last_peered': '2024-09-11T01:24:16.217857+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:16.217857+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3022848, 'num_objects': 738, 'num_object_clones': 0, 'num_object_copies': 1476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 738, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17461, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3409}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'19995", 'reported_seq': 20285, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.776126+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:18.776126+0000', 'last_peered': '2024-09-11T01:24:18.776126+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:18.776126+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'9900", 'ondisk_log_start': "41'9900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2666496, 'num_objects': 651, 'num_object_clones': 0, 'num_object_copies': 1302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16388, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3466}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'19896", 'reported_seq': 20201, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.858739+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:18.858739+0000', 'last_peered': '2024-09-11T01:24:18.858739+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:18.858739+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2883584, 'num_objects': 704, 'num_object_clones': 0, 'num_object_copies': 1408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16431, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3361}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:21.218 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'19468", 'reported_seq': 19713, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:18.816268+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:18.816268+0000', 'last_peered': '2024-09-11T01:24:18.816268+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:18.816268+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'9400", 'ondisk_log_start': "41'9400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2745286, 'num_objects': 672, 'num_object_clones': 0, 'num_object_copies': 1344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 672, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16125, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3430}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:21.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:21.965 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:21.966 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36132/22962 objects degraded (157.356%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":662,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":11481,"data_bytes":46977801,"bytes_used":1581604864,"bytes_avail":288328687616,"bytes_total":289910292480,"degraded_objects":36132,"degraded_total":22962,"degraded_ratio":1.5735563104259211,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":496},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:21.995 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:21.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:22.596 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:22.598 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:22.620 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-11T01:24:20.646386+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":46977801,"num_objects":11481,"num_object_clones":0,"num_object_copies":22962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11481,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":273764,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13503,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160802,"ondisk_log_size":160802,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544536,"kb_used_data":294440,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281570984,"statfs":{"total":289910292480,"available":288328687616,"internally_reserved":0,"allocated":301506560,"data_stored":300589361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1280035847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,10,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24428544,"num_objects":-5964,"num_object_clones":0,"num_object_copies":-11928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5964,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005324"},"pg_stats":[{"pgid":"2.f","version":"45'19524","reported_seq":20969,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.785511+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:18.785511+0000","last_peered":"2024-09-11T01:24:18.785511+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:18.785511+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19524,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2726}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19749","reported_seq":19835,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.184151+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:16.184151+0000","last_peered":"2024-09-11T01:24:16.184151+0000","last_clean":"2024-09-11T01:24:16.184151+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:16.184151+0000","last_undegraded":"2024-09-11T01:24:16.184151+0000","last_fullsized":"2024-09-11T01:24:16.184151+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3167174,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17847,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19285","reported_seq":19455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.294993+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:16.294993+0000","last_peered":"2024-09-11T01:24:16.294993+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:16.294993+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9200","ondisk_log_start":"41'9200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3112984,"num_objects":761,"num_object_clones":0,"num_object_copies":1522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":761,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17346,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3202}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19606","reported_seq":19657,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.262905+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:16.262905+0000","last_peered":"2024-09-11T01:24:16.262905+0000","last_clean":"2024-09-11T01:24:16.262905+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:16.262905+0000","last_undegraded":"2024-09-11T01:24:16.262905+0000","last_fullsized":"2024-09-11T01:24:16.262905+0000","mapping_epoch":31,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3096576,"num_objects":756,"num_object_clones":0,"num_object_copies":1512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19606,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19575","reported_seq":19702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:18.792050+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:18.792050+0000","last_peered":"2024-09-11T01:24:18.792050+0000","last_clean":"2024-09-11T01:24:18.792050+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:18.792050+0000","last_undegraded":"2024-09-11T01:24:18.792050+0000","last_fullsized":"2024-09-11T01:24:18.792050+0000","mapping_epoch":31,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2875875,"num_objects":703,"num_object_clones":0,"num_object_copies":1406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16373,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19543","reported_seq":19815,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.810644+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:18.810644+0000","last_peered":"2024-09-11T01:24:18.810644+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:18.810644+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2720227,"num_objects":665,"num_object_clones":0,"num_object_copies":1330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":665,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16031,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3384},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20119","reported_seq":20412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.746616+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:18.746616+0000","last_peered":"2024-09-11T01:24:18.746616+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:18.746616+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2839494,"num_objects":695,"num_object_clones":0,"num_object_copies":1390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":695,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16872,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3511},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19714","reported_seq":19839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:18.697441+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:18.697441+0000","last_peered":"2024-09-11T01:24:18.697441+0000","last_clean":"2024-09-11T01:24:18.697441+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:18.697441+0000","last_undegraded":"2024-09-11T01:24:18.697441+0000","last_fullsized":"2024-09-11T01:24:18.697441+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2908160,"num_objects":710,"num_object_clones":0,"num_object_copies":1420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16667,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19507","reported_seq":27855,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:17.235758+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:17.235758+0000","last_peered":"2024-09-11T01:24:17.235758+0000","last_clean":"2024-09-11T01:24:17.235758+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:17.235758+0000","last_undegraded":"2024-09-11T01:24:17.235758+0000","last_fullsized":"2024-09-11T01:24:17.235758+0000","mapping_epoch":40,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3039715,"num_objects":743,"num_object_clones":0,"num_object_copies":1486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":743,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17374,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'19876","reported_seq":20159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.829449+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:18.829449+0000","last_peered":"2024-09-11T01:24:18.829449+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:18.829449+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2940928,"num_objects":718,"num_object_clones":0,"num_object_copies":1436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16240,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19365","reported_seq":27528,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:16.197953+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:16.197953+0000","last_peered":"2024-09-11T01:24:16.197953+0000","last_clean":"2024-09-11T01:24:16.197953+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:16.197953+0000","last_undegraded":"2024-09-11T01:24:16.197953+0000","last_fullsized":"2024-09-11T01:24:16.197953+0000","mapping_epoch":44,"log_start":"41'9300","ondisk_log_start":"41'9300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3170787,"num_objects":775,"num_object_clones":0,"num_object_copies":1550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17134,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19824","reported_seq":19983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:16.217857+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:16.217857+0000","last_peered":"2024-09-11T01:24:16.217857+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:16.217857+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3022848,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17461,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3409}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19653","reported_seq":28127,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:17.218815+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:17.218815+0000","last_peered":"2024-09-11T01:24:17.218815+0000","last_clean":"2024-09-11T01:24:17.218815+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:17.218815+0000","last_undegraded":"2024-09-11T01:24:17.218815+0000","last_fullsized":"2024-09-11T01:24:17.218815+0000","mapping_epoch":42,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2863104,"num_objects":699,"num_object_clones":0,"num_object_copies":1398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16343,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'19995","reported_seq":20285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.776126+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:18.776126+0000","last_peered":"2024-09-11T01:24:18.776126+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:18.776126+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2666496,"num_objects":651,"num_object_clones":0,"num_object_copies":1302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16388,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'19896","reported_seq":20201,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.858739+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:18.858739+0000","last_peered":"2024-09-11T01:24:18.858739+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:18.858739+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2883584,"num_objects":704,"num_object_clones":0,"num_object_copies":1408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16431,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3361}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19468","reported_seq":19713,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:18.816268+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:18.816268+0000","last_peered":"2024-09-11T01:24:18.816268+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:18.816268+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2745286,"num_objects":672,"num_object_clones":0,"num_object_copies":1344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16125,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3430},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":46977782,"num_objects":11479,"num_object_clones":0,"num_object_copies":22958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11479,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":273762,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13502,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298323968,"data_stored":298171304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493200,"kb_used_data":158848,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93878640,"statfs":{"total":96636764160,"available":96131727360,"internally_reserved":0,"allocated":162660352,"data_stored":162368023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790060,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501640,"kb_used_data":36792,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93870200,"statfs":{"total":96636764160,"available":96123084800,"internally_reserved":0,"allocated":37675008,"data_stored":37342472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549696,"kb_used_data":98800,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93822144,"statfs":{"total":96636764160,"available":96073875456,"internally_reserved":0,"allocated":101171200,"data_stored":100878866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,7,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":100741120,"data_stored":100704990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36130816,"data_stored":36054484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":161452032,"data_stored":161411830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:25.622 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:26.215 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:26.216 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:26.242 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-11T01:24:24.648203+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":37454601,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":276089,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13573,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160927,"ondisk_log_size":160927,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536484,"kb_used_data":273904,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281579036,"statfs":{"total":289910292480,"available":288336932864,"internally_reserved":0,"allocated":280477696,"data_stored":279560497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1292815367},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,8,5,8,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24334336,"num_objects":-5941,"num_object_clones":0,"num_object_copies":-11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5941,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005447"},"pg_stats":[{"pgid":"2.f","version":"45'19686","reported_seq":21272,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.882720+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:23.882720+0000","last_peered":"2024-09-11T01:24:23.882720+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:23.882720+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19686,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'19885","reported_seq":19970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:21.239653+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:21.239653+0000","last_peered":"2024-09-11T01:24:21.239653+0000","last_clean":"2024-09-11T01:24:21.239653+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:21.239653+0000","last_undegraded":"2024-09-11T01:24:21.239653+0000","last_fullsized":"2024-09-11T01:24:21.239653+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2610118,"num_objects":639,"num_object_clones":0,"num_object_copies":1278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17983,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19454","reported_seq":19628,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:21.297929+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:21.297929+0000","last_peered":"2024-09-11T01:24:21.297929+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:21.297929+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2420760,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17515,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19749","reported_seq":19800,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:21.316119+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:21.316119+0000","last_peered":"2024-09-11T01:24:21.316119+0000","last_clean":"2024-09-11T01:24:21.316119+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:21.316119+0000","last_undegraded":"2024-09-11T01:24:21.316119+0000","last_fullsized":"2024-09-11T01:24:21.316119+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2510848,"num_objects":613,"num_object_clones":0,"num_object_copies":1226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19749,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19725","reported_seq":19852,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:23.808881+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:23.808881+0000","last_peered":"2024-09-11T01:24:23.808881+0000","last_clean":"2024-09-11T01:24:23.808881+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:23.808881+0000","last_undegraded":"2024-09-11T01:24:23.808881+0000","last_fullsized":"2024-09-11T01:24:23.808881+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2261475,"num_objects":553,"num_object_clones":0,"num_object_copies":1106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16523,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19679","reported_seq":19948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.840552+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:23.840552+0000","last_peered":"2024-09-11T01:24:23.840552+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:23.840552+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2163171,"num_objects":529,"num_object_clones":0,"num_object_copies":1058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16167,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3520},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20249","reported_seq":20542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.854297+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:23.854297+0000","last_peered":"2024-09-11T01:24:23.854297+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:23.854297+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2307014,"num_objects":565,"num_object_clones":0,"num_object_copies":1130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17002,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3641},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19862","reported_seq":19986,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:23.886162+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:23.886162+0000","last_peered":"2024-09-11T01:24:23.886162+0000","last_clean":"2024-09-11T01:24:23.886162+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:23.886162+0000","last_undegraded":"2024-09-11T01:24:23.886162+0000","last_fullsized":"2024-09-11T01:24:23.886162+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301952,"num_objects":562,"num_object_clones":0,"num_object_copies":1124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16815,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19656","reported_seq":28005,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:22.164456+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:22.164456+0000","last_peered":"2024-09-11T01:24:22.164456+0000","last_clean":"2024-09-11T01:24:22.164456+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:22.164456+0000","last_undegraded":"2024-09-11T01:24:22.164456+0000","last_fullsized":"2024-09-11T01:24:22.164456+0000","mapping_epoch":40,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429411,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17523,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20031","reported_seq":20315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.759922+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:23.759922+0000","last_peered":"2024-09-11T01:24:23.759922+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:23.759922+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306048,"num_objects":563,"num_object_clones":0,"num_object_copies":1126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16395,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3467}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19484","reported_seq":27647,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:21.283114+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:21.283114+0000","last_peered":"2024-09-11T01:24:21.283114+0000","last_clean":"2024-09-11T01:24:21.283114+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:21.283114+0000","last_undegraded":"2024-09-11T01:24:21.283114+0000","last_fullsized":"2024-09-11T01:24:21.283114+0000","mapping_epoch":44,"log_start":"41'9400","ondisk_log_start":"41'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2683363,"num_objects":656,"num_object_clones":0,"num_object_copies":1312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":656,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17253,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'19965","reported_seq":20124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:21.283327+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:21.283327+0000","last_peered":"2024-09-11T01:24:21.283327+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:21.283327+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2445312,"num_objects":597,"num_object_clones":0,"num_object_copies":1194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17602,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3550}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19796","reported_seq":28269,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:22.176046+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:22.176046+0000","last_peered":"2024-09-11T01:24:22.176046+0000","last_clean":"2024-09-11T01:24:22.176046+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:22.176046+0000","last_undegraded":"2024-09-11T01:24:22.176046+0000","last_fullsized":"2024-09-11T01:24:22.176046+0000","mapping_epoch":42,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2277376,"num_objects":556,"num_object_clones":0,"num_object_copies":1112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16486,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20124","reported_seq":20414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.829520+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:23.829520+0000","last_peered":"2024-09-11T01:24:23.829520+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:23.829520+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112,"num_objects":522,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16517,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20065","reported_seq":20370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.870183+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:23.870183+0000","last_peered":"2024-09-11T01:24:23.870183+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:23.870183+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2191360,"num_objects":535,"num_object_clones":0,"num_object_copies":1070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19614","reported_seq":19859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.776720+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:23.776720+0000","last_peered":"2024-09-11T01:24:23.776720+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:23.776720+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2147270,"num_objects":526,"num_object_clones":0,"num_object_copies":1052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16271,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3576},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":37454582,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":276087,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13572,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":279207936,"data_stored":279055272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160924,"ondisk_log_size":160924,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492020,"kb_used_data":153508,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93879820,"statfs":{"total":96636764160,"available":96132935680,"internally_reserved":0,"allocated":157192192,"data_stored":156899863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790061,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497620,"kb_used_data":28612,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93874220,"statfs":{"total":96636764160,"available":96127201280,"internally_reserved":0,"allocated":29298688,"data_stored":28966152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546844,"kb_used_data":91784,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93824996,"statfs":{"total":96636764160,"available":96076795904,"internally_reserved":0,"allocated":93986816,"data_stored":93694482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93585408,"data_stored":93549278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28856320,"data_stored":28779988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":156766208,"data_stored":156726006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:26.244 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:26.858 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:26.860 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:26.885 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-11T01:24:26.649117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":34517769,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":276806,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13573,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535928,"kb_used_data":269188,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281579592,"statfs":{"total":289910292480,"available":288337502208,"internally_reserved":0,"allocated":275648512,"data_stored":274731313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1297075207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,5,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22007808,"num_objects":-5373,"num_object_clones":0,"num_object_copies":-10746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5373,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005439"},"pg_stats":[{"pgid":"2.f","version":"45'19686","reported_seq":21272,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.882720+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:23.882720+0000","last_peered":"2024-09-11T01:24:23.882720+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:23.882720+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19686,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20034","reported_seq":20120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.323282+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:26.323282+0000","last_peered":"2024-09-11T01:24:26.323282+0000","last_clean":"2024-09-11T01:24:26.323282+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:26.323282+0000","last_undegraded":"2024-09-11T01:24:26.323282+0000","last_fullsized":"2024-09-11T01:24:26.323282+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999814,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18132,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19585","reported_seq":19759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.136737+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:26.136737+0000","last_peered":"2024-09-11T01:24:26.136737+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:26.136737+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884184,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17646,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19887","reported_seq":19938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.270577+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:26.270577+0000","last_peered":"2024-09-11T01:24:26.270577+0000","last_clean":"2024-09-11T01:24:26.270577+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:26.270577+0000","last_undegraded":"2024-09-11T01:24:26.270577+0000","last_fullsized":"2024-09-11T01:24:26.270577+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1945600,"num_objects":475,"num_object_clones":0,"num_object_copies":950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19887,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19725","reported_seq":19852,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:23.808881+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:23.808881+0000","last_peered":"2024-09-11T01:24:23.808881+0000","last_clean":"2024-09-11T01:24:23.808881+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:23.808881+0000","last_undegraded":"2024-09-11T01:24:23.808881+0000","last_fullsized":"2024-09-11T01:24:23.808881+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2261475,"num_objects":553,"num_object_clones":0,"num_object_copies":1106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16523,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19679","reported_seq":19948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.840552+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:23.840552+0000","last_peered":"2024-09-11T01:24:23.840552+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:23.840552+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2163171,"num_objects":529,"num_object_clones":0,"num_object_copies":1058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16167,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3520},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20249","reported_seq":20542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.854297+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:23.854297+0000","last_peered":"2024-09-11T01:24:23.854297+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:23.854297+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2307014,"num_objects":565,"num_object_clones":0,"num_object_copies":1130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17002,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3641},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19862","reported_seq":19986,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:23.886162+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:23.886162+0000","last_peered":"2024-09-11T01:24:23.886162+0000","last_clean":"2024-09-11T01:24:23.886162+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:23.886162+0000","last_undegraded":"2024-09-11T01:24:23.886162+0000","last_fullsized":"2024-09-11T01:24:23.886162+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301952,"num_objects":562,"num_object_clones":0,"num_object_copies":1124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16815,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19656","reported_seq":28005,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:22.164456+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:22.164456+0000","last_peered":"2024-09-11T01:24:22.164456+0000","last_clean":"2024-09-11T01:24:22.164456+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:22.164456+0000","last_undegraded":"2024-09-11T01:24:22.164456+0000","last_fullsized":"2024-09-11T01:24:22.164456+0000","mapping_epoch":40,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429411,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17523,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20031","reported_seq":20315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.759922+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:23.759922+0000","last_peered":"2024-09-11T01:24:23.759922+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:23.759922+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306048,"num_objects":563,"num_object_clones":0,"num_object_copies":1126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16395,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3467}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19635","reported_seq":27797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.284369+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:26.284369+0000","last_peered":"2024-09-11T01:24:26.284369+0000","last_clean":"2024-09-11T01:24:26.284369+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:26.284369+0000","last_undegraded":"2024-09-11T01:24:26.284369+0000","last_fullsized":"2024-09-11T01:24:26.284369+0000","mapping_epoch":44,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064867,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17404,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20113","reported_seq":20269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.308010+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:26.308010+0000","last_peered":"2024-09-11T01:24:26.308010+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:26.308010+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17750,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19796","reported_seq":28269,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:22.176046+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:22.176046+0000","last_peered":"2024-09-11T01:24:22.176046+0000","last_clean":"2024-09-11T01:24:22.176046+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:22.176046+0000","last_undegraded":"2024-09-11T01:24:22.176046+0000","last_fullsized":"2024-09-11T01:24:22.176046+0000","mapping_epoch":42,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2277376,"num_objects":556,"num_object_clones":0,"num_object_copies":1112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16486,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20124","reported_seq":20414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.829520+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:23.829520+0000","last_peered":"2024-09-11T01:24:23.829520+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:23.829520+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112,"num_objects":522,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16517,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20065","reported_seq":20370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.870183+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:23.870183+0000","last_peered":"2024-09-11T01:24:23.870183+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:23.870183+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2191360,"num_objects":535,"num_object_clones":0,"num_object_copies":1070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19614","reported_seq":19859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.776720+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:23.776720+0000","last_peered":"2024-09-11T01:24:23.776720+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:23.776720+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2147270,"num_objects":526,"num_object_clones":0,"num_object_copies":1052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16271,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3576},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":34517750,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":276804,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13572,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274374656,"data_stored":274221992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491464,"kb_used_data":148792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93880376,"statfs":{"total":96636764160,"available":96133505024,"internally_reserved":0,"allocated":152363008,"data_stored":152070679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790061,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497620,"kb_used_data":28612,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93874220,"statfs":{"total":96636764160,"available":96127201280,"internally_reserved":0,"allocated":29298688,"data_stored":28966152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546844,"kb_used_data":91784,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93824996,"statfs":{"total":96636764160,"available":96076795904,"internally_reserved":0,"allocated":93986816,"data_stored":93694482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93585408,"data_stored":93549278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28856320,"data_stored":28779988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151932928,"data_stored":151892726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'19686", 'reported_seq': 21272, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.882720+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:23.882720+0000', 'last_peered': '2024-09-11T01:24:23.882720+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:23.882720+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'9600", 'ondisk_log_start': "41'9600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2260992, 'num_objects': 552, 'num_object_clones': 0, 'num_object_copies': 1104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3370, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19686, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 753, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2818}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'19585", 'reported_seq': 19759, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:26.136737+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:26.136737+0000', 'last_peered': '2024-09-11T01:24:26.136737+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:26.136737+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1884184, 'num_objects': 461, 'num_object_clones': 0, 'num_object_copies': 922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 461, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17646, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'19679", 'reported_seq': 19948, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.840552+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:23.840552+0000', 'last_peered': '2024-09-11T01:24:23.840552+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:23.840552+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'9600", 'ondisk_log_start': "41'9600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2163171, 'num_objects': 529, 'num_object_clones': 0, 'num_object_copies': 1058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 529, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16167, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3520}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.886 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20249", 'reported_seq': 20542, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.854297+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:23.854297+0000', 'last_peered': '2024-09-11T01:24:23.854297+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:23.854297+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2307014, 'num_objects': 565, 'num_object_clones': 0, 'num_object_copies': 1130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 565, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17002, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3641}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20031", 'reported_seq': 20315, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.759922+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:23.759922+0000', 'last_peered': '2024-09-11T01:24:23.759922+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:23.759922+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2306048, 'num_objects': 563, 'num_object_clones': 0, 'num_object_copies': 1126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 563, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16395, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3467}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20113", 'reported_seq': 20269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:26.308010+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:26.308010+0000', 'last_peered': '2024-09-11T01:24:26.308010+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:26.308010+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1839104, 'num_objects': 449, 'num_object_clones': 0, 'num_object_copies': 898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17750, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3698}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20124", 'reported_seq': 20414, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.829520+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:23.829520+0000', 'last_peered': '2024-09-11T01:24:23.829520+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:23.829520+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2138112, 'num_objects': 522, 'num_object_clones': 0, 'num_object_copies': 1044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16517, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3595}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20065", 'reported_seq': 20370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.870183+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:23.870183+0000', 'last_peered': '2024-09-11T01:24:23.870183+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:23.870183+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2191360, 'num_objects': 535, 'num_object_clones': 0, 'num_object_copies': 1070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16600, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:26.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'19614", 'reported_seq': 19859, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:23.776720+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:23.776720+0000', 'last_peered': '2024-09-11T01:24:23.776720+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:23.776720+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'9600", 'ondisk_log_start': "41'9600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2147270, 'num_objects': 526, 'num_object_clones': 0, 'num_object_copies': 1052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 526, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16271, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3576}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.888 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:27.620 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:27.620 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36062/16878 objects degraded (213.663%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":668,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":8439,"data_bytes":34517769,"bytes_used":1572790272,"bytes_avail":288337502208,"bytes_total":289910292480,"degraded_objects":36062,"degraded_total":16878,"degraded_ratio":2.1366275625074063,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":447},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:27.643 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:27.644 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:28.226 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:28.228 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:28.249 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-11T01:24:26.649117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":34517769,"num_objects":8439,"num_object_clones":0,"num_object_copies":16878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8439,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":276806,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13573,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160844,"ondisk_log_size":160844,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535928,"kb_used_data":269188,"kb_used_omap":12,"kb_used_meta":1266675,"kb_avail":281579592,"statfs":{"total":289910292480,"available":288337502208,"internally_reserved":0,"allocated":275648512,"data_stored":274731313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1297075207},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,5,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22007808,"num_objects":-5373,"num_object_clones":0,"num_object_copies":-10746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5373,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":153,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005439"},"pg_stats":[{"pgid":"2.f","version":"45'19686","reported_seq":21272,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.882720+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:23.882720+0000","last_peered":"2024-09-11T01:24:23.882720+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:23.882720+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19686,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":753,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20034","reported_seq":20120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.323282+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:26.323282+0000","last_peered":"2024-09-11T01:24:26.323282+0000","last_clean":"2024-09-11T01:24:26.323282+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:26.323282+0000","last_undegraded":"2024-09-11T01:24:26.323282+0000","last_fullsized":"2024-09-11T01:24:26.323282+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999814,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18132,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19585","reported_seq":19759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.136737+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:26.136737+0000","last_peered":"2024-09-11T01:24:26.136737+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:26.136737+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884184,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17646,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19887","reported_seq":19938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.270577+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:26.270577+0000","last_peered":"2024-09-11T01:24:26.270577+0000","last_clean":"2024-09-11T01:24:26.270577+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:26.270577+0000","last_undegraded":"2024-09-11T01:24:26.270577+0000","last_fullsized":"2024-09-11T01:24:26.270577+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1945600,"num_objects":475,"num_object_clones":0,"num_object_copies":950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19887,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19725","reported_seq":19852,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:23.808881+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:23.808881+0000","last_peered":"2024-09-11T01:24:23.808881+0000","last_clean":"2024-09-11T01:24:23.808881+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:23.808881+0000","last_undegraded":"2024-09-11T01:24:23.808881+0000","last_fullsized":"2024-09-11T01:24:23.808881+0000","mapping_epoch":31,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2261475,"num_objects":553,"num_object_clones":0,"num_object_copies":1106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":553,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16523,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19679","reported_seq":19948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.840552+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:23.840552+0000","last_peered":"2024-09-11T01:24:23.840552+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:23.840552+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2163171,"num_objects":529,"num_object_clones":0,"num_object_copies":1058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16167,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3520},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20249","reported_seq":20542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.854297+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:23.854297+0000","last_peered":"2024-09-11T01:24:23.854297+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:23.854297+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2307014,"num_objects":565,"num_object_clones":0,"num_object_copies":1130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17002,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3641},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'19862","reported_seq":19986,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:23.886162+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:23.886162+0000","last_peered":"2024-09-11T01:24:23.886162+0000","last_clean":"2024-09-11T01:24:23.886162+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:23.886162+0000","last_undegraded":"2024-09-11T01:24:23.886162+0000","last_fullsized":"2024-09-11T01:24:23.886162+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2301952,"num_objects":562,"num_object_clones":0,"num_object_copies":1124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16815,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19656","reported_seq":28005,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:22.164456+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:22.164456+0000","last_peered":"2024-09-11T01:24:22.164456+0000","last_clean":"2024-09-11T01:24:22.164456+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:22.164456+0000","last_undegraded":"2024-09-11T01:24:22.164456+0000","last_fullsized":"2024-09-11T01:24:22.164456+0000","mapping_epoch":40,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2429411,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17523,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20031","reported_seq":20315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.759922+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:23.759922+0000","last_peered":"2024-09-11T01:24:23.759922+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:23.759922+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2306048,"num_objects":563,"num_object_clones":0,"num_object_copies":1126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16395,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3467}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19635","reported_seq":27797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.284369+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:26.284369+0000","last_peered":"2024-09-11T01:24:26.284369+0000","last_clean":"2024-09-11T01:24:26.284369+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:26.284369+0000","last_undegraded":"2024-09-11T01:24:26.284369+0000","last_fullsized":"2024-09-11T01:24:26.284369+0000","mapping_epoch":44,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064867,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17404,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20113","reported_seq":20269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.308010+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:26.308010+0000","last_peered":"2024-09-11T01:24:26.308010+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:26.308010+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17750,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19796","reported_seq":28269,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:22.176046+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:22.176046+0000","last_peered":"2024-09-11T01:24:22.176046+0000","last_clean":"2024-09-11T01:24:22.176046+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:22.176046+0000","last_undegraded":"2024-09-11T01:24:22.176046+0000","last_fullsized":"2024-09-11T01:24:22.176046+0000","mapping_epoch":42,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2277376,"num_objects":556,"num_object_clones":0,"num_object_copies":1112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16486,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20124","reported_seq":20414,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.829520+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:23.829520+0000","last_peered":"2024-09-11T01:24:23.829520+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:23.829520+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2138112,"num_objects":522,"num_object_clones":0,"num_object_copies":1044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16517,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3595}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20065","reported_seq":20370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.870183+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:23.870183+0000","last_peered":"2024-09-11T01:24:23.870183+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:23.870183+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2191360,"num_objects":535,"num_object_clones":0,"num_object_copies":1070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16600,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19614","reported_seq":19859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:23.776720+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:23.776720+0000","last_peered":"2024-09-11T01:24:23.776720+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:23.776720+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2147270,"num_objects":526,"num_object_clones":0,"num_object_copies":1052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":526,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16271,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3576},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":34517750,"num_objects":8437,"num_object_clones":0,"num_object_copies":16874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36062,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8437,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":276804,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13572,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":274374656,"data_stored":274221992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160841,"ondisk_log_size":160841,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491464,"kb_used_data":148792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93880376,"statfs":{"total":96636764160,"available":96133505024,"internally_reserved":0,"allocated":152363008,"data_stored":152070679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790061,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497620,"kb_used_data":28612,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93874220,"statfs":{"total":96636764160,"available":96127201280,"internally_reserved":0,"allocated":29298688,"data_stored":28966152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546844,"kb_used_data":91784,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93824996,"statfs":{"total":96636764160,"available":96076795904,"internally_reserved":0,"allocated":93986816,"data_stored":93694482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":93585408,"data_stored":93549278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28856320,"data_stored":28779988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151932928,"data_stored":151892726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:31.252 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:31.815 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:31.817 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:31.840 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-11T01:24:30.650930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27935497,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":278413,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13644,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541288,"kb_used_data":256500,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281574232,"statfs":{"total":289910292480,"available":288332013568,"internally_reserved":0,"allocated":262656000,"data_stored":261738801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1315556359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,8,13,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24223744,"num_objects":-5914,"num_object_clones":0,"num_object_copies":-11828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005431"},"pg_stats":[{"pgid":"2.f","version":"45'19854","reported_seq":21580,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.893888+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:28.893888+0000","last_peered":"2024-09-11T01:24:28.893888+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:28.893888+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19854,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":824,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2915}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20034","reported_seq":20120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.323282+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:26.323282+0000","last_peered":"2024-09-11T01:24:26.323282+0000","last_clean":"2024-09-11T01:24:26.323282+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:26.323282+0000","last_undegraded":"2024-09-11T01:24:26.323282+0000","last_fullsized":"2024-09-11T01:24:26.323282+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999814,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18132,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19585","reported_seq":19759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.136737+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:26.136737+0000","last_peered":"2024-09-11T01:24:26.136737+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:26.136737+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884184,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17646,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19887","reported_seq":19938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.270577+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:26.270577+0000","last_peered":"2024-09-11T01:24:26.270577+0000","last_clean":"2024-09-11T01:24:26.270577+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:26.270577+0000","last_undegraded":"2024-09-11T01:24:26.270577+0000","last_fullsized":"2024-09-11T01:24:26.270577+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1945600,"num_objects":475,"num_object_clones":0,"num_object_copies":950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19887,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19886","reported_seq":20013,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:28.865337+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:28.865337+0000","last_peered":"2024-09-11T01:24:28.865337+0000","last_clean":"2024-09-11T01:24:28.865337+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:28.865337+0000","last_undegraded":"2024-09-11T01:24:28.865337+0000","last_fullsized":"2024-09-11T01:24:28.865337+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602019,"num_objects":392,"num_object_clones":0,"num_object_copies":784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16684,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19812","reported_seq":20084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.843483+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:28.843483+0000","last_peered":"2024-09-11T01:24:28.843483+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:28.843483+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618403,"num_objects":396,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16300,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3653},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20399","reported_seq":20692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.888540+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:28.888540+0000","last_peered":"2024-09-11T01:24:28.888540+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:28.888540+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1692614,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17152,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3791},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20010","reported_seq":20135,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:28.810817+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:28.810817+0000","last_peered":"2024-09-11T01:24:28.810817+0000","last_clean":"2024-09-11T01:24:28.810817+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:28.810817+0000","last_undegraded":"2024-09-11T01:24:28.810817+0000","last_fullsized":"2024-09-11T01:24:28.810817+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695744,"num_objects":414,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19799","reported_seq":28148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:27.213904+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:27.213904+0000","last_peered":"2024-09-11T01:24:27.213904+0000","last_clean":"2024-09-11T01:24:27.213904+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:27.213904+0000","last_undegraded":"2024-09-11T01:24:27.213904+0000","last_fullsized":"2024-09-11T01:24:27.213904+0000","mapping_epoch":40,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1843683,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17666,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20185","reported_seq":20470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.806832+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:28.806832+0000","last_peered":"2024-09-11T01:24:28.806832+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:28.806832+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1675264,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19635","reported_seq":27797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.284369+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:26.284369+0000","last_peered":"2024-09-11T01:24:26.284369+0000","last_clean":"2024-09-11T01:24:26.284369+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:26.284369+0000","last_undegraded":"2024-09-11T01:24:26.284369+0000","last_fullsized":"2024-09-11T01:24:26.284369+0000","mapping_epoch":44,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064867,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17404,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20113","reported_seq":20269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.308010+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:26.308010+0000","last_peered":"2024-09-11T01:24:26.308010+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:26.308010+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17750,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19936","reported_seq":28412,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:27.119515+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:27.119515+0000","last_peered":"2024-09-11T01:24:27.119515+0000","last_clean":"2024-09-11T01:24:27.119515+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:27.119515+0000","last_undegraded":"2024-09-11T01:24:27.119515+0000","last_fullsized":"2024-09-11T01:24:27.119515+0000","mapping_epoch":42,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":416,"num_object_clones":0,"num_object_copies":832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20248","reported_seq":20538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.830463+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:28.830463+0000","last_peered":"2024-09-11T01:24:28.830463+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:28.830463+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16641,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20200","reported_seq":20505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.833169+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:28.833169+0000","last_peered":"2024-09-11T01:24:28.833169+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:28.833169+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19765","reported_seq":20010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.914935+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:28.914935+0000","last_peered":"2024-09-11T01:24:28.914935+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:28.914935+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1528774,"num_objects":375,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16422,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3727},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":27935478,"num_objects":6830,"num_object_clones":0,"num_object_copies":13660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6830,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":278411,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13643,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":260050944,"data_stored":259898280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160948,"ondisk_log_size":160948,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491464,"kb_used_data":148792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93880376,"statfs":{"total":96636764160,"available":96133505024,"internally_reserved":0,"allocated":152363008,"data_stored":152070679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790062,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518156,"kb_used_data":22780,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93853684,"statfs":{"total":96636764160,"available":96106172416,"internally_reserved":0,"allocated":23326720,"data_stored":22994184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507244207},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531668,"kb_used_data":84928,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93840172,"statfs":{"total":96636764160,"available":96092336128,"internally_reserved":0,"allocated":86966272,"data_stored":86673938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86429696,"data_stored":86393566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21688320,"data_stored":21611988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151932928,"data_stored":151892726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:31.841 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:32.444 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:32.447 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:32.468 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-11T01:24:30.650930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27935497,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":278413,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13644,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1541288,"kb_used_data":256500,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281574232,"statfs":{"total":289910292480,"available":288332013568,"internally_reserved":0,"allocated":262656000,"data_stored":261738801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1315556359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,8,13,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24223744,"num_objects":-5914,"num_object_clones":0,"num_object_copies":-11828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-224,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5914,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005431"},"pg_stats":[{"pgid":"2.f","version":"45'19854","reported_seq":21580,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.893888+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:28.893888+0000","last_peered":"2024-09-11T01:24:28.893888+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:28.893888+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19854,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":824,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2915}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20034","reported_seq":20120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.323282+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:26.323282+0000","last_peered":"2024-09-11T01:24:26.323282+0000","last_clean":"2024-09-11T01:24:26.323282+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:26.323282+0000","last_undegraded":"2024-09-11T01:24:26.323282+0000","last_fullsized":"2024-09-11T01:24:26.323282+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1999814,"num_objects":490,"num_object_clones":0,"num_object_copies":980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":490,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18132,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19585","reported_seq":19759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.136737+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:26.136737+0000","last_peered":"2024-09-11T01:24:26.136737+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:26.136737+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9500","ondisk_log_start":"41'9500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1884184,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17646,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'19887","reported_seq":19938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.270577+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:26.270577+0000","last_peered":"2024-09-11T01:24:26.270577+0000","last_clean":"2024-09-11T01:24:26.270577+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:26.270577+0000","last_undegraded":"2024-09-11T01:24:26.270577+0000","last_fullsized":"2024-09-11T01:24:26.270577+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1945600,"num_objects":475,"num_object_clones":0,"num_object_copies":950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19887,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19886","reported_seq":20013,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:28.865337+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:28.865337+0000","last_peered":"2024-09-11T01:24:28.865337+0000","last_clean":"2024-09-11T01:24:28.865337+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:28.865337+0000","last_undegraded":"2024-09-11T01:24:28.865337+0000","last_fullsized":"2024-09-11T01:24:28.865337+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602019,"num_objects":392,"num_object_clones":0,"num_object_copies":784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16684,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19812","reported_seq":20084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.843483+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:28.843483+0000","last_peered":"2024-09-11T01:24:28.843483+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:28.843483+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618403,"num_objects":396,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16300,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3653},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20399","reported_seq":20692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.888540+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:28.888540+0000","last_peered":"2024-09-11T01:24:28.888540+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:28.888540+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1692614,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17152,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3791},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20010","reported_seq":20135,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:28.810817+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:28.810817+0000","last_peered":"2024-09-11T01:24:28.810817+0000","last_clean":"2024-09-11T01:24:28.810817+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:28.810817+0000","last_undegraded":"2024-09-11T01:24:28.810817+0000","last_fullsized":"2024-09-11T01:24:28.810817+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695744,"num_objects":414,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19799","reported_seq":28148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:27.213904+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:27.213904+0000","last_peered":"2024-09-11T01:24:27.213904+0000","last_clean":"2024-09-11T01:24:27.213904+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:27.213904+0000","last_undegraded":"2024-09-11T01:24:27.213904+0000","last_fullsized":"2024-09-11T01:24:27.213904+0000","mapping_epoch":40,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1843683,"num_objects":451,"num_object_clones":0,"num_object_copies":902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17666,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20185","reported_seq":20470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.806832+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:28.806832+0000","last_peered":"2024-09-11T01:24:28.806832+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:28.806832+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1675264,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19635","reported_seq":27797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:26.284369+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:26.284369+0000","last_peered":"2024-09-11T01:24:26.284369+0000","last_clean":"2024-09-11T01:24:26.284369+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:26.284369+0000","last_undegraded":"2024-09-11T01:24:26.284369+0000","last_fullsized":"2024-09-11T01:24:26.284369+0000","mapping_epoch":44,"log_start":"41'9600","ondisk_log_start":"41'9600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2064867,"num_objects":505,"num_object_clones":0,"num_object_copies":1010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":505,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17404,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20113","reported_seq":20269,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:26.308010+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:26.308010+0000","last_peered":"2024-09-11T01:24:26.308010+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:26.308010+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1839104,"num_objects":449,"num_object_clones":0,"num_object_copies":898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17750,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3698}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'19936","reported_seq":28412,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:27.119515+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:27.119515+0000","last_peered":"2024-09-11T01:24:27.119515+0000","last_clean":"2024-09-11T01:24:27.119515+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:27.119515+0000","last_undegraded":"2024-09-11T01:24:27.119515+0000","last_fullsized":"2024-09-11T01:24:27.119515+0000","mapping_epoch":42,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703936,"num_objects":416,"num_object_clones":0,"num_object_copies":832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16626,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20248","reported_seq":20538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.830463+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:28.830463+0000","last_peered":"2024-09-11T01:24:28.830463+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:28.830463+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16641,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20200","reported_seq":20505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.833169+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:28.833169+0000","last_peered":"2024-09-11T01:24:28.833169+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:28.833169+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19765","reported_seq":20010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.914935+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:28.914935+0000","last_peered":"2024-09-11T01:24:28.914935+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:28.914935+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1528774,"num_objects":375,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16422,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3727},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":27935478,"num_objects":6830,"num_object_clones":0,"num_object_copies":13660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6830,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":278411,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13643,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":260050944,"data_stored":259898280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160948,"ondisk_log_size":160948,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491464,"kb_used_data":148792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93880376,"statfs":{"total":96636764160,"available":96133505024,"internally_reserved":0,"allocated":152363008,"data_stored":152070679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790062,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518156,"kb_used_data":22780,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93853684,"statfs":{"total":96636764160,"available":96106172416,"internally_reserved":0,"allocated":23326720,"data_stored":22994184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507244207},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531668,"kb_used_data":84928,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93840172,"statfs":{"total":96636764160,"available":96092336128,"internally_reserved":0,"allocated":86966272,"data_stored":86673938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86429696,"data_stored":86393566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":21688320,"data_stored":21611988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":151932928,"data_stored":151892726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:32.469 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'19854", 'reported_seq': 21580, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.893888+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:28.893888+0000', 'last_peered': '2024-09-11T01:24:28.893888+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:28.893888+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1572864, 'num_objects': 384, 'num_object_clones': 0, 'num_object_copies': 768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3299, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 384, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19854, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 824, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2915}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'19585", 'reported_seq': 19759, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:26.136737+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:26.136737+0000', 'last_peered': '2024-09-11T01:24:26.136737+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:26.136737+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'9500", 'ondisk_log_start': "41'9500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1884184, 'num_objects': 461, 'num_object_clones': 0, 'num_object_copies': 922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 461, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17646, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'19812", 'reported_seq': 20084, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.843483+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:28.843483+0000', 'last_peered': '2024-09-11T01:24:28.843483+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:28.843483+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1618403, 'num_objects': 396, 'num_object_clones': 0, 'num_object_copies': 792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 396, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16300, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3653}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20399", 'reported_seq': 20692, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.888540+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:28.888540+0000', 'last_peered': '2024-09-11T01:24:28.888540+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:28.888540+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10300", 'ondisk_log_start': "41'10300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1692614, 'num_objects': 415, 'num_object_clones': 0, 'num_object_copies': 830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 415, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17152, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3791}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20185", 'reported_seq': 20470, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.806832+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:28.806832+0000', 'last_peered': '2024-09-11T01:24:28.806832+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:28.806832+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1675264, 'num_objects': 409, 'num_object_clones': 0, 'num_object_copies': 818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16549, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3621}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20113", 'reported_seq': 20269, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:26.308010+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:26.308010+0000', 'last_peered': '2024-09-11T01:24:26.308010+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:26.308010+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1839104, 'num_objects': 449, 'num_object_clones': 0, 'num_object_copies': 898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17750, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3698}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20248", 'reported_seq': 20538, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.830463+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:28.830463+0000', 'last_peered': '2024-09-11T01:24:28.830463+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:28.830463+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1630208, 'num_objects': 398, 'num_object_clones': 0, 'num_object_copies': 796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 398, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16641, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3719}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20200", 'reported_seq': 20505, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.833169+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:28.833169+0000', 'last_peered': '2024-09-11T01:24:28.833169+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:28.833169+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1638400, 'num_objects': 400, 'num_object_clones': 0, 'num_object_copies': 800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 400, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16735, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3665}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:32.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'19765", 'reported_seq': 20010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.914935+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:28.914935+0000', 'last_peered': '2024-09-11T01:24:28.914935+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:28.914935+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'9700", 'ondisk_log_start': "41'9700", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1528774, 'num_objects': 375, 'num_object_clones': 0, 'num_object_copies': 750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 375, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16422, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3727}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:32.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:33.212 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:33.212 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35991/11524 objects degraded (312.313%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":673,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5762,"data_bytes":23552777,"bytes_used":1556406272,"bytes_avail":288353886208,"bytes_total":289910292480,"degraded_objects":35991,"degraded_total":11524,"degraded_ratio":3.1231343283582089,"recovering_objects_per_sec":11,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:33.237 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:33.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:33.814 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:33.815 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:33.840 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-11T01:24:32.651817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":23552777,"num_objects":5762,"num_object_clones":0,"num_object_copies":11524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5762,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":279483,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13644,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161021,"ondisk_log_size":161021,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1519928,"kb_used_data":243456,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281595592,"statfs":{"total":289910292480,"available":288353886208,"internally_reserved":0,"allocated":249298944,"data_stored":248381745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,9,17,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23425024,"num_objects":-5719,"num_object_clones":0,"num_object_copies":-11438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5719,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":141,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005431"},"pg_stats":[{"pgid":"2.f","version":"45'19854","reported_seq":21580,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.893888+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:28.893888+0000","last_peered":"2024-09-11T01:24:28.893888+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:28.893888+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1572864,"num_objects":384,"num_object_clones":0,"num_object_copies":768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3299,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19854,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":824,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2915}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20191","reported_seq":20277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:31.287951+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:31.287951+0000","last_peered":"2024-09-11T01:24:31.287951+0000","last_clean":"2024-09-11T01:24:31.287951+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:31.287951+0000","last_undegraded":"2024-09-11T01:24:31.287951+0000","last_fullsized":"2024-09-11T01:24:31.287951+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1356742,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18289,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19758","reported_seq":19930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:31.304361+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:31.304361+0000","last_peered":"2024-09-11T01:24:31.304361+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:31.304361+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175576,"num_objects":288,"num_object_clones":0,"num_object_copies":576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":288,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17819,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3675}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20032","reported_seq":20083,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:31.255243+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:31.255243+0000","last_peered":"2024-09-11T01:24:31.255243+0000","last_clean":"2024-09-11T01:24:31.255243+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:31.255243+0000","last_undegraded":"2024-09-11T01:24:31.255243+0000","last_fullsized":"2024-09-11T01:24:31.255243+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1351680,"num_objects":330,"num_object_clones":0,"num_object_copies":660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20032,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'19886","reported_seq":20013,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:28.865337+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:28.865337+0000","last_peered":"2024-09-11T01:24:28.865337+0000","last_clean":"2024-09-11T01:24:28.865337+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:28.865337+0000","last_undegraded":"2024-09-11T01:24:28.865337+0000","last_fullsized":"2024-09-11T01:24:28.865337+0000","mapping_epoch":31,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1602019,"num_objects":392,"num_object_clones":0,"num_object_copies":784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16684,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19812","reported_seq":20084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.843483+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:28.843483+0000","last_peered":"2024-09-11T01:24:28.843483+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:28.843483+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1618403,"num_objects":396,"num_object_clones":0,"num_object_copies":792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":396,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16300,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3653},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20399","reported_seq":20692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.888540+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:28.888540+0000","last_peered":"2024-09-11T01:24:28.888540+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:28.888540+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1692614,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17152,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3791},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20010","reported_seq":20135,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:28.810817+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:28.810817+0000","last_peered":"2024-09-11T01:24:28.810817+0000","last_clean":"2024-09-11T01:24:28.810817+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:28.810817+0000","last_undegraded":"2024-09-11T01:24:28.810817+0000","last_fullsized":"2024-09-11T01:24:28.810817+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1695744,"num_objects":414,"num_object_clones":0,"num_object_copies":828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16963,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19933","reported_seq":28282,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:32.346810+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:32.346810+0000","last_peered":"2024-09-11T01:24:32.346810+0000","last_clean":"2024-09-11T01:24:32.346810+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:32.346810+0000","last_undegraded":"2024-09-11T01:24:32.346810+0000","last_fullsized":"2024-09-11T01:24:32.346810+0000","mapping_epoch":40,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294819,"num_objects":317,"num_object_clones":0,"num_object_copies":634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17800,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20185","reported_seq":20470,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.806832+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:28.806832+0000","last_peered":"2024-09-11T01:24:28.806832+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:28.806832+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1675264,"num_objects":409,"num_object_clones":0,"num_object_copies":818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16549,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19799","reported_seq":27962,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:31.227380+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:31.227380+0000","last_peered":"2024-09-11T01:24:31.227380+0000","last_clean":"2024-09-11T01:24:31.227380+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:31.227380+0000","last_undegraded":"2024-09-11T01:24:31.227380+0000","last_fullsized":"2024-09-11T01:24:31.227380+0000","mapping_epoch":44,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1393123,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17568,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20265","reported_seq":20425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:31.307576+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:31.307576+0000","last_peered":"2024-09-11T01:24:31.307576+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:31.307576+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1216512,"num_objects":297,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17902,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3850}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20081","reported_seq":28557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:32.346541+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:32.346541+0000","last_peered":"2024-09-11T01:24:32.346541+0000","last_clean":"2024-09-11T01:24:32.346541+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:32.346541+0000","last_undegraded":"2024-09-11T01:24:32.346541+0000","last_fullsized":"2024-09-11T01:24:32.346541+0000","mapping_epoch":42,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1110016,"num_objects":271,"num_object_clones":0,"num_object_copies":542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16771,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20248","reported_seq":20538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.830463+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:28.830463+0000","last_peered":"2024-09-11T01:24:28.830463+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:28.830463+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16641,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20200","reported_seq":20505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.833169+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:28.833169+0000","last_peered":"2024-09-11T01:24:28.833169+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:28.833169+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1638400,"num_objects":400,"num_object_clones":0,"num_object_copies":800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":400,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19765","reported_seq":20010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.914935+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:28.914935+0000","last_peered":"2024-09-11T01:24:28.914935+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:28.914935+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9700","ondisk_log_start":"41'9700","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1528774,"num_objects":375,"num_object_clones":0,"num_object_copies":750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16422,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3727},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":23552758,"num_objects":5760,"num_object_clones":0,"num_object_copies":11520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5760,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":279481,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13643,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":247808000,"data_stored":247655336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161018,"ondisk_log_size":161018,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490928,"kb_used_data":144096,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93880912,"statfs":{"total":96636764160,"available":96134053888,"internally_reserved":0,"allocated":147554304,"data_stored":147261975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,6,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790063,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497332,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93874508,"statfs":{"total":96636764160,"available":96127496192,"internally_reserved":0,"allocated":14778368,"data_stored":14445832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531668,"kb_used_data":84928,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93840172,"statfs":{"total":96636764160,"available":96092336128,"internally_reserved":0,"allocated":86966272,"data_stored":86673938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":86429696,"data_stored":86393566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14307328,"data_stored":14230996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":147070976,"data_stored":147030774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:36.842 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:37.425 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:37.428 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:37.451 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-11T01:24:36.653651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15319817,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":281493,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13724,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500104,"kb_used_data":231956,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281615416,"statfs":{"total":289910292480,"available":288374185984,"internally_reserved":0,"allocated":237522944,"data_stored":236605745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,3,7,12,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22134784,"num_objects":-5404,"num_object_clones":0,"num_object_copies":-10808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5404,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005447"},"pg_stats":[{"pgid":"2.f","version":"45'20018","reported_seq":21906,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.964679+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:33.964679+0000","last_peered":"2024-09-11T01:24:33.964679+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:33.964679+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":901120,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20018,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20338","reported_seq":20424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.333553+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:36.333553+0000","last_peered":"2024-09-11T01:24:36.333553+0000","last_clean":"2024-09-11T01:24:36.333553+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:36.333553+0000","last_undegraded":"2024-09-11T01:24:36.333553+0000","last_fullsized":"2024-09-11T01:24:36.333553+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754630,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18436,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19888","reported_seq":20062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:36.253672+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:36.253672+0000","last_peered":"2024-09-11T01:24:36.253672+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:36.253672+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643096,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17949,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3805}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20179","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.283974+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:36.283974+0000","last_peered":"2024-09-11T01:24:36.283974+0000","last_clean":"2024-09-11T01:24:36.283974+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:36.283974+0000","last_undegraded":"2024-09-11T01:24:36.283974+0000","last_fullsized":"2024-09-11T01:24:36.283974+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20179,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20015","reported_seq":20142,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:33.892689+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:33.892689+0000","last_peered":"2024-09-11T01:24:33.892689+0000","last_clean":"2024-09-11T01:24:33.892689+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:33.892689+0000","last_undegraded":"2024-09-11T01:24:33.892689+0000","last_fullsized":"2024-09-11T01:24:33.892689+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073635,"num_objects":263,"num_object_clones":0,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16813,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19959","reported_seq":20229,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.953324+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:33.953324+0000","last_peered":"2024-09-11T01:24:33.953324+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:33.953324+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016291,"num_objects":249,"num_object_clones":0,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16447,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3800},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20547","reported_seq":20840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.777912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:33.777912+0000","last_peered":"2024-09-11T01:24:33.777912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:33.777912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1086406,"num_objects":267,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":267,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17300,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3939},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20148","reported_seq":20273,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:33.928930+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:33.928930+0000","last_peered":"2024-09-11T01:24:33.928930+0000","last_clean":"2024-09-11T01:24:33.928930+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:33.928930+0000","last_undegraded":"2024-09-11T01:24:33.928930+0000","last_fullsized":"2024-09-11T01:24:33.928930+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":276,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17101,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19933","reported_seq":28282,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:32.346810+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:32.346810+0000","last_peered":"2024-09-11T01:24:32.346810+0000","last_clean":"2024-09-11T01:24:32.346810+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:32.346810+0000","last_undegraded":"2024-09-11T01:24:32.346810+0000","last_fullsized":"2024-09-11T01:24:32.346810+0000","mapping_epoch":40,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294819,"num_objects":317,"num_object_clones":0,"num_object_copies":634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17800,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20338","reported_seq":20623,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.776643+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:33.776643+0000","last_peered":"2024-09-11T01:24:33.776643+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:33.776643+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16702,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3774}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19955","reported_seq":28117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.339468+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:36.339468+0000","last_peered":"2024-09-11T01:24:36.339468+0000","last_clean":"2024-09-11T01:24:36.339468+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:36.339468+0000","last_undegraded":"2024-09-11T01:24:36.339468+0000","last_fullsized":"2024-09-11T01:24:36.339468+0000","mapping_epoch":44,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754147,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17724,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20403","reported_seq":20563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:36.320956+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:36.320956+0000","last_peered":"2024-09-11T01:24:36.320956+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:36.320956+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651264,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18040,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20081","reported_seq":28557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:32.346541+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:32.346541+0000","last_peered":"2024-09-11T01:24:32.346541+0000","last_clean":"2024-09-11T01:24:32.346541+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:32.346541+0000","last_undegraded":"2024-09-11T01:24:32.346541+0000","last_fullsized":"2024-09-11T01:24:32.346541+0000","mapping_epoch":42,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1110016,"num_objects":271,"num_object_clones":0,"num_object_copies":542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16771,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20374","reported_seq":20662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.930589+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:33.930589+0000","last_peered":"2024-09-11T01:24:33.930589+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:33.930589+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":272,"num_object_clones":0,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16767,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20355","reported_seq":20660,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.896687+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:33.896687+0000","last_peered":"2024-09-11T01:24:33.896687+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:33.896687+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1003520,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16890,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19897","reported_seq":20142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.955359+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:33.955359+0000","last_peered":"2024-09-11T01:24:33.955359+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:33.955359+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":988102,"num_objects":243,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16554,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3859},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":15319798,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":281491,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13723,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":235814912,"data_stored":235662248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473812,"kb_used_data":139460,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93898028,"statfs":{"total":96636764160,"available":96151580672,"internally_reserved":0,"allocated":142807040,"data_stored":142514711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790063,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497332,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93874508,"statfs":{"total":96636764160,"available":96127496192,"internally_reserved":0,"allocated":14778368,"data_stored":14445832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528960,"kb_used_data":78064,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93842880,"statfs":{"total":96636764160,"available":96095109120,"internally_reserved":0,"allocated":79937536,"data_stored":79645202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79298560,"data_stored":79262430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14307328,"data_stored":14230996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142209024,"data_stored":142168822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:37.453 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:38.066 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:38.066 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:38.091 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-11T01:24:36.653651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15319817,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":281493,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13724,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500104,"kb_used_data":231956,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281615416,"statfs":{"total":289910292480,"available":288374185984,"internally_reserved":0,"allocated":237522944,"data_stored":236605745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,3,7,12,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22134784,"num_objects":-5404,"num_object_clones":0,"num_object_copies":-10808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5404,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005447"},"pg_stats":[{"pgid":"2.f","version":"45'20018","reported_seq":21906,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.964679+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:33.964679+0000","last_peered":"2024-09-11T01:24:33.964679+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:33.964679+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":901120,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20018,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20338","reported_seq":20424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.333553+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:36.333553+0000","last_peered":"2024-09-11T01:24:36.333553+0000","last_clean":"2024-09-11T01:24:36.333553+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:36.333553+0000","last_undegraded":"2024-09-11T01:24:36.333553+0000","last_fullsized":"2024-09-11T01:24:36.333553+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754630,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18436,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19888","reported_seq":20062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:36.253672+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:36.253672+0000","last_peered":"2024-09-11T01:24:36.253672+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:36.253672+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643096,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17949,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3805}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20179","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.283974+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:36.283974+0000","last_peered":"2024-09-11T01:24:36.283974+0000","last_clean":"2024-09-11T01:24:36.283974+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:36.283974+0000","last_undegraded":"2024-09-11T01:24:36.283974+0000","last_fullsized":"2024-09-11T01:24:36.283974+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20179,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20015","reported_seq":20142,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:33.892689+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:33.892689+0000","last_peered":"2024-09-11T01:24:33.892689+0000","last_clean":"2024-09-11T01:24:33.892689+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:33.892689+0000","last_undegraded":"2024-09-11T01:24:33.892689+0000","last_fullsized":"2024-09-11T01:24:33.892689+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073635,"num_objects":263,"num_object_clones":0,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16813,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19959","reported_seq":20229,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.953324+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:33.953324+0000","last_peered":"2024-09-11T01:24:33.953324+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:33.953324+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016291,"num_objects":249,"num_object_clones":0,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16447,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3800},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20547","reported_seq":20840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.777912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:33.777912+0000","last_peered":"2024-09-11T01:24:33.777912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:33.777912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1086406,"num_objects":267,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":267,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17300,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3939},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20148","reported_seq":20273,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:33.928930+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:33.928930+0000","last_peered":"2024-09-11T01:24:33.928930+0000","last_clean":"2024-09-11T01:24:33.928930+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:33.928930+0000","last_undegraded":"2024-09-11T01:24:33.928930+0000","last_fullsized":"2024-09-11T01:24:33.928930+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":276,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17101,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'19933","reported_seq":28282,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:32.346810+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:32.346810+0000","last_peered":"2024-09-11T01:24:32.346810+0000","last_clean":"2024-09-11T01:24:32.346810+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:32.346810+0000","last_undegraded":"2024-09-11T01:24:32.346810+0000","last_fullsized":"2024-09-11T01:24:32.346810+0000","mapping_epoch":40,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294819,"num_objects":317,"num_object_clones":0,"num_object_copies":634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17800,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20338","reported_seq":20623,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.776643+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:33.776643+0000","last_peered":"2024-09-11T01:24:33.776643+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:33.776643+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16702,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3774}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19955","reported_seq":28117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.339468+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:36.339468+0000","last_peered":"2024-09-11T01:24:36.339468+0000","last_clean":"2024-09-11T01:24:36.339468+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:36.339468+0000","last_undegraded":"2024-09-11T01:24:36.339468+0000","last_fullsized":"2024-09-11T01:24:36.339468+0000","mapping_epoch":44,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754147,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17724,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20403","reported_seq":20563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:36.320956+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:36.320956+0000","last_peered":"2024-09-11T01:24:36.320956+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:36.320956+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651264,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18040,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20081","reported_seq":28557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:32.346541+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:32.346541+0000","last_peered":"2024-09-11T01:24:32.346541+0000","last_clean":"2024-09-11T01:24:32.346541+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:32.346541+0000","last_undegraded":"2024-09-11T01:24:32.346541+0000","last_fullsized":"2024-09-11T01:24:32.346541+0000","mapping_epoch":42,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1110016,"num_objects":271,"num_object_clones":0,"num_object_copies":542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16771,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20374","reported_seq":20662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.930589+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:33.930589+0000","last_peered":"2024-09-11T01:24:33.930589+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:33.930589+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":272,"num_object_clones":0,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16767,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20355","reported_seq":20660,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.896687+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:33.896687+0000","last_peered":"2024-09-11T01:24:33.896687+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:33.896687+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1003520,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16890,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19897","reported_seq":20142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.955359+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:33.955359+0000","last_peered":"2024-09-11T01:24:33.955359+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:33.955359+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":988102,"num_objects":243,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16554,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3859},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":15319798,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":281491,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13723,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":235814912,"data_stored":235662248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473812,"kb_used_data":139460,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93898028,"statfs":{"total":96636764160,"available":96151580672,"internally_reserved":0,"allocated":142807040,"data_stored":142514711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790063,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497332,"kb_used_data":14432,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93874508,"statfs":{"total":96636764160,"available":96127496192,"internally_reserved":0,"allocated":14778368,"data_stored":14445832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528960,"kb_used_data":78064,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93842880,"statfs":{"total":96636764160,"available":96095109120,"internally_reserved":0,"allocated":79937536,"data_stored":79645202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79298560,"data_stored":79262430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14307328,"data_stored":14230996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142209024,"data_stored":142168822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'20018", 'reported_seq': 21906, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.964679+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:33.964679+0000', 'last_peered': '2024-09-11T01:24:33.964679+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:33.964679+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 901120, 'num_objects': 220, 'num_object_clones': 0, 'num_object_copies': 440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3219, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20018, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 904, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2999}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'19888", 'reported_seq': 20062, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:36.253672+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:36.253672+0000', 'last_peered': '2024-09-11T01:24:36.253672+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:36.253672+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 643096, 'num_objects': 158, 'num_object_clones': 0, 'num_object_copies': 316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 158, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17949, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'19959", 'reported_seq': 20229, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.953324+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:33.953324+0000', 'last_peered': '2024-09-11T01:24:33.953324+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:33.953324+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'9900", 'ondisk_log_start': "41'9900", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1016291, 'num_objects': 249, 'num_object_clones': 0, 'num_object_copies': 498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 249, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16447, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3800}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:38.093 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20547", 'reported_seq': 20840, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.777912+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:33.777912+0000', 'last_peered': '2024-09-11T01:24:33.777912+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:33.777912+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1086406, 'num_objects': 267, 'num_object_clones': 0, 'num_object_copies': 534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 267, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17300, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3939}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20338", 'reported_seq': 20623, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.776643+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:33.776643+0000', 'last_peered': '2024-09-11T01:24:33.776643+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:33.776643+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10300", 'ondisk_log_start': "41'10300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1048576, 'num_objects': 256, 'num_object_clones': 0, 'num_object_copies': 512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16702, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3774}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20403", 'reported_seq': 20563, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:36.320956+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:36.320956+0000', 'last_peered': '2024-09-11T01:24:36.320956+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:36.320956+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10400", 'ondisk_log_start': "41'10400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 651264, 'num_objects': 159, 'num_object_clones': 0, 'num_object_copies': 318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18040, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3988}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20374", 'reported_seq': 20662, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.930589+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:33.930589+0000', 'last_peered': '2024-09-11T01:24:33.930589+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:33.930589+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10300", 'ondisk_log_start': "41'10300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1114112, 'num_objects': 272, 'num_object_clones': 0, 'num_object_copies': 544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 272, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16767, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3845}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20355", 'reported_seq': 20660, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.896687+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:33.896687+0000', 'last_peered': '2024-09-11T01:24:33.896687+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:33.896687+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10300", 'ondisk_log_start': "41'10300", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1003520, 'num_objects': 245, 'num_object_clones': 0, 'num_object_copies': 490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 245, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16890, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3820}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:38.094 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'19897", 'reported_seq': 20142, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.955359+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:33.955359+0000', 'last_peered': '2024-09-11T01:24:33.955359+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:33.955359+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'9800", 'ondisk_log_start': "41'9800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 988102, 'num_objects': 243, 'num_object_clones': 0, 'num_object_copies': 486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 243, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16554, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3859}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:38.095 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:38.851 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:38.851 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35911/6882 objects degraded (521.811%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":679,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3441,"data_bytes":14045961,"bytes_used":1537343488,"bytes_avail":288372948992,"bytes_total":289910292480,"degraded_objects":35911,"degraded_total":6882,"degraded_ratio":5.2181052019761696,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":416},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:38.875 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:38.876 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:39.484 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:39.485 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:39.518 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-11T01:24:38.654565+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":14045961,"num_objects":3441,"num_object_clones":0,"num_object_copies":6882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3441,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":281804,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13724,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501312,"kb_used_data":224844,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281614208,"statfs":{"total":289910292480,"available":288372948992,"internally_reserved":0,"allocated":230240256,"data_stored":229323057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,3,13,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20471808,"num_objects":-4998,"num_object_clones":0,"num_object_copies":-9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4998,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":151,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005447"},"pg_stats":[{"pgid":"2.f","version":"45'20018","reported_seq":21906,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.964679+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:33.964679+0000","last_peered":"2024-09-11T01:24:33.964679+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:33.964679+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":901120,"num_objects":220,"num_object_clones":0,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20018,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20338","reported_seq":20424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.333553+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:36.333553+0000","last_peered":"2024-09-11T01:24:36.333553+0000","last_clean":"2024-09-11T01:24:36.333553+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:36.333553+0000","last_undegraded":"2024-09-11T01:24:36.333553+0000","last_fullsized":"2024-09-11T01:24:36.333553+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754630,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18436,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'19888","reported_seq":20062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:36.253672+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:36.253672+0000","last_peered":"2024-09-11T01:24:36.253672+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:36.253672+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":643096,"num_objects":158,"num_object_clones":0,"num_object_copies":316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17949,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3805}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20179","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.283974+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:36.283974+0000","last_peered":"2024-09-11T01:24:36.283974+0000","last_clean":"2024-09-11T01:24:36.283974+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:36.283974+0000","last_undegraded":"2024-09-11T01:24:36.283974+0000","last_fullsized":"2024-09-11T01:24:36.283974+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20179,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20015","reported_seq":20142,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:33.892689+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:33.892689+0000","last_peered":"2024-09-11T01:24:33.892689+0000","last_clean":"2024-09-11T01:24:33.892689+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:33.892689+0000","last_undegraded":"2024-09-11T01:24:33.892689+0000","last_fullsized":"2024-09-11T01:24:33.892689+0000","mapping_epoch":31,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073635,"num_objects":263,"num_object_clones":0,"num_object_copies":526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16813,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'19959","reported_seq":20229,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.953324+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:33.953324+0000","last_peered":"2024-09-11T01:24:33.953324+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:33.953324+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1016291,"num_objects":249,"num_object_clones":0,"num_object_copies":498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16447,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3800},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20547","reported_seq":20840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.777912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:33.777912+0000","last_peered":"2024-09-11T01:24:33.777912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:33.777912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1086406,"num_objects":267,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":267,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17300,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3939},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20148","reported_seq":20273,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:33.928930+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:33.928930+0000","last_peered":"2024-09-11T01:24:33.928930+0000","last_clean":"2024-09-11T01:24:33.928930+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:33.928930+0000","last_undegraded":"2024-09-11T01:24:33.928930+0000","last_fullsized":"2024-09-11T01:24:33.928930+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1130496,"num_objects":276,"num_object_clones":0,"num_object_copies":552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17101,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20104","reported_seq":28453,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:37.418552+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:37.418552+0000","last_peered":"2024-09-11T01:24:37.418552+0000","last_clean":"2024-09-11T01:24:37.418552+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:37.418552+0000","last_undegraded":"2024-09-11T01:24:37.418552+0000","last_fullsized":"2024-09-11T01:24:37.418552+0000","mapping_epoch":40,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":594403,"num_objects":146,"num_object_clones":0,"num_object_copies":292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":146,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17971,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20338","reported_seq":20623,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.776643+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:33.776643+0000","last_peered":"2024-09-11T01:24:33.776643+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:33.776643+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1048576,"num_objects":256,"num_object_clones":0,"num_object_copies":512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16702,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3774}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":103,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843378+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843378+0000","last_peered":"2024-09-11T01:23:46.843378+0000","last_clean":"2024-09-11T01:23:46.843378+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843378+0000","last_undegraded":"2024-09-11T01:23:46.843378+0000","last_fullsized":"2024-09-11T01:23:46.843378+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'19955","reported_seq":28117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:36.339468+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:36.339468+0000","last_peered":"2024-09-11T01:24:36.339468+0000","last_clean":"2024-09-11T01:24:36.339468+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:36.339468+0000","last_undegraded":"2024-09-11T01:24:36.339468+0000","last_fullsized":"2024-09-11T01:24:36.339468+0000","mapping_epoch":44,"log_start":"41'9900","ondisk_log_start":"41'9900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":754147,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17724,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20403","reported_seq":20563,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:36.320956+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:36.320956+0000","last_peered":"2024-09-11T01:24:36.320956+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:36.320956+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651264,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18040,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3988}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20221","reported_seq":28697,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:37.473257+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:37.473257+0000","last_peered":"2024-09-11T01:24:37.473257+0000","last_clean":"2024-09-11T01:24:37.473257+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:37.473257+0000","last_undegraded":"2024-09-11T01:24:37.473257+0000","last_fullsized":"2024-09-11T01:24:37.473257+0000","mapping_epoch":42,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":536576,"num_objects":131,"num_object_clones":0,"num_object_copies":262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16911,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20374","reported_seq":20662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.930589+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:33.930589+0000","last_peered":"2024-09-11T01:24:33.930589+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:33.930589+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114112,"num_objects":272,"num_object_clones":0,"num_object_copies":544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16767,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20355","reported_seq":20660,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.896687+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:33.896687+0000","last_peered":"2024-09-11T01:24:33.896687+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:33.896687+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1003520,"num_objects":245,"num_object_clones":0,"num_object_copies":490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16890,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3820}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'19897","reported_seq":20142,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.955359+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:33.955359+0000","last_peered":"2024-09-11T01:24:33.955359+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:33.955359+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'9800","ondisk_log_start":"41'9800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":988102,"num_objects":243,"num_object_clones":0,"num_object_copies":486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16554,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3859},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":14045942,"num_objects":3439,"num_object_clones":0,"num_object_copies":6878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35911,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3439,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":281802,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13723,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":228474880,"data_stored":228322216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473812,"kb_used_data":139460,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93898028,"statfs":{"total":96636764160,"available":96151580672,"internally_reserved":0,"allocated":142807040,"data_stored":142514711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":342355630},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790064,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498540,"kb_used_data":7320,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93873300,"statfs":{"total":96636764160,"available":96126259200,"internally_reserved":0,"allocated":7495680,"data_stored":7163144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,0,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528960,"kb_used_data":78064,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93842880,"statfs":{"total":96636764160,"available":96095109120,"internally_reserved":0,"allocated":79937536,"data_stored":79645202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":461696682},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":79298560,"data_stored":79262430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":6890964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":142209024,"data_stored":142168822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:42.372 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Removed 163042 objects 2024-09-11T01:24:42.372 INFO:tasks.lost_unfound.radosbench.client.admin.smithi039.stdout:Clean up completed and total clean up time :365.998 2024-09-11T01:24:42.520 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:42.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:42.837 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:42.849 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T01:24:42.656178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4416265,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":284155,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479840,"kb_used_data":207524,"kb_used_omap":12,"kb_used_meta":1272243,"kb_avail":281635680,"statfs":{"total":289910292480,"available":288394936320,"internally_reserved":0,"allocated":212504576,"data_stored":211587377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1302776839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,13,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23519232,"num_objects":-5742,"num_object_clones":0,"num_object_copies":-11484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005248"},"pg_stats":[{"pgid":"2.f","version":"45'20151","reported_seq":22211,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.027016+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:39.027016+0000","last_peered":"2024-09-11T01:24:39.027016+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:39.027016+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356352,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20151,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":990,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20493","reported_seq":20579,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.319148+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:41.319148+0000","last_peered":"2024-09-11T01:24:41.319148+0000","last_clean":"2024-09-11T01:24:41.319148+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:41.319148+0000","last_undegraded":"2024-09-11T01:24:41.319148+0000","last_fullsized":"2024-09-11T01:24:41.319148+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119750,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18591,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20016","reported_seq":20190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:41.301827+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:41.301827+0000","last_peered":"2024-09-11T01:24:41.301827+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:41.301827+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118808,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18077,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20326","reported_seq":20377,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.348070+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:41.348070+0000","last_peered":"2024-09-11T01:24:41.348070+0000","last_clean":"2024-09-11T01:24:41.348070+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:41.348070+0000","last_undegraded":"2024-09-11T01:24:41.348070+0000","last_fullsized":"2024-09-11T01:24:41.348070+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20326,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20180","reported_seq":20306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:39.055400+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:39.055400+0000","last_peered":"2024-09-11T01:24:39.055400+0000","last_clean":"2024-09-11T01:24:39.055400+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:39.055400+0000","last_undegraded":"2024-09-11T01:24:39.055400+0000","last_fullsized":"2024-09-11T01:24:39.055400+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397795,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16978,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20104","reported_seq":20376,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.997388+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:38.997388+0000","last_peered":"2024-09-11T01:24:38.997388+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:38.997388+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422371,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16592,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3945},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20699","reported_seq":20992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.957013+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:38.957013+0000","last_peered":"2024-09-11T01:24:38.957013+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:38.957013+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463814,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17452,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4091},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20309","reported_seq":20434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:38.992921+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:38.992921+0000","last_peered":"2024-09-11T01:24:38.992921+0000","last_clean":"2024-09-11T01:24:38.992921+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:38.992921+0000","last_undegraded":"2024-09-11T01:24:38.992921+0000","last_fullsized":"2024-09-11T01:24:38.992921+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471040,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17262,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20489","reported_seq":20774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.017024+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:39.017024+0000","last_peered":"2024-09-11T01:24:39.017024+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:39.017024+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430080,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20108","reported_seq":28271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.197743+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:41.197743+0000","last_peered":"2024-09-11T01:24:41.197743+0000","last_clean":"2024-09-11T01:24:41.197743+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:41.197743+0000","last_undegraded":"2024-09-11T01:24:41.197743+0000","last_fullsized":"2024-09-11T01:24:41.197743+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127459,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17877,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20538","reported_seq":20698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:41.336409+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:41.336409+0000","last_peered":"2024-09-11T01:24:41.336409+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:41.336409+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18175,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20537","reported_seq":20827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.025555+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:39.025555+0000","last_peered":"2024-09-11T01:24:39.025555+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:39.025555+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16930,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20495","reported_seq":20800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.980770+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:38.980770+0000","last_peered":"2024-09-11T01:24:38.980770+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:38.980770+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430080,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17030,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3960}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20044","reported_seq":20289,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.016658+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:39.016658+0000","last_peered":"2024-09-11T01:24:39.016658+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:39.016658+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385990,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16701,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4006},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4416246,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":284153,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13809,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":209403904,"data_stored":209255312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473272,"kb_used_data":134760,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93898568,"statfs":{"total":96636764160,"available":96152133632,"internally_reserved":0,"allocated":137994240,"data_stored":137701911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790065,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480536,"kb_used_data":1792,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93891304,"statfs":{"total":96636764160,"available":96144695296,"internally_reserved":0,"allocated":1835008,"data_stored":1502472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526032,"kb_used_data":70972,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93845808,"statfs":{"total":96636764160,"available":96098107392,"internally_reserved":0,"allocated":72675328,"data_stored":72382994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":71979008,"data_stored":71942878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137342976,"data_stored":137302774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:42.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:43.160 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:43.161 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:43.173 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T01:24:42.656178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4416265,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":284155,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479840,"kb_used_data":207524,"kb_used_omap":12,"kb_used_meta":1272243,"kb_avail":281635680,"statfs":{"total":289910292480,"available":288394936320,"internally_reserved":0,"allocated":212504576,"data_stored":211587377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1302776839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,13,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23519232,"num_objects":-5742,"num_object_clones":0,"num_object_copies":-11484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005248"},"pg_stats":[{"pgid":"2.f","version":"45'20151","reported_seq":22211,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.027016+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:39.027016+0000","last_peered":"2024-09-11T01:24:39.027016+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:39.027016+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356352,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20151,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":990,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20493","reported_seq":20579,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.319148+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:41.319148+0000","last_peered":"2024-09-11T01:24:41.319148+0000","last_clean":"2024-09-11T01:24:41.319148+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:41.319148+0000","last_undegraded":"2024-09-11T01:24:41.319148+0000","last_fullsized":"2024-09-11T01:24:41.319148+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119750,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18591,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20016","reported_seq":20190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:41.301827+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:41.301827+0000","last_peered":"2024-09-11T01:24:41.301827+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:41.301827+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118808,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18077,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20326","reported_seq":20377,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.348070+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:41.348070+0000","last_peered":"2024-09-11T01:24:41.348070+0000","last_clean":"2024-09-11T01:24:41.348070+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:41.348070+0000","last_undegraded":"2024-09-11T01:24:41.348070+0000","last_fullsized":"2024-09-11T01:24:41.348070+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20326,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20180","reported_seq":20306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:39.055400+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:39.055400+0000","last_peered":"2024-09-11T01:24:39.055400+0000","last_clean":"2024-09-11T01:24:39.055400+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:39.055400+0000","last_undegraded":"2024-09-11T01:24:39.055400+0000","last_fullsized":"2024-09-11T01:24:39.055400+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397795,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16978,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20104","reported_seq":20376,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.997388+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:38.997388+0000","last_peered":"2024-09-11T01:24:38.997388+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:38.997388+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422371,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16592,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3945},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20699","reported_seq":20992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.957013+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:38.957013+0000","last_peered":"2024-09-11T01:24:38.957013+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:38.957013+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463814,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17452,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4091},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20309","reported_seq":20434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:38.992921+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:38.992921+0000","last_peered":"2024-09-11T01:24:38.992921+0000","last_clean":"2024-09-11T01:24:38.992921+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:38.992921+0000","last_undegraded":"2024-09-11T01:24:38.992921+0000","last_fullsized":"2024-09-11T01:24:38.992921+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471040,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17262,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20489","reported_seq":20774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.017024+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:39.017024+0000","last_peered":"2024-09-11T01:24:39.017024+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:39.017024+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430080,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20108","reported_seq":28271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.197743+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:41.197743+0000","last_peered":"2024-09-11T01:24:41.197743+0000","last_clean":"2024-09-11T01:24:41.197743+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:41.197743+0000","last_undegraded":"2024-09-11T01:24:41.197743+0000","last_fullsized":"2024-09-11T01:24:41.197743+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127459,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17877,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20538","reported_seq":20698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:41.336409+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:41.336409+0000","last_peered":"2024-09-11T01:24:41.336409+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:41.336409+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18175,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20537","reported_seq":20827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.025555+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:39.025555+0000","last_peered":"2024-09-11T01:24:39.025555+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:39.025555+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16930,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20495","reported_seq":20800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.980770+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:38.980770+0000","last_peered":"2024-09-11T01:24:38.980770+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:38.980770+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430080,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17030,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3960}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20044","reported_seq":20289,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.016658+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:39.016658+0000","last_peered":"2024-09-11T01:24:39.016658+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:39.016658+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385990,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16701,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4006},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4416246,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":284153,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13809,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":209403904,"data_stored":209255312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473272,"kb_used_data":134760,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93898568,"statfs":{"total":96636764160,"available":96152133632,"internally_reserved":0,"allocated":137994240,"data_stored":137701911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790065,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480536,"kb_used_data":1792,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93891304,"statfs":{"total":96636764160,"available":96144695296,"internally_reserved":0,"allocated":1835008,"data_stored":1502472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526032,"kb_used_data":70972,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93845808,"statfs":{"total":96636764160,"available":96098107392,"internally_reserved":0,"allocated":72675328,"data_stored":72382994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":71979008,"data_stored":71942878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137342976,"data_stored":137302774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'20151", 'reported_seq': 22211, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:39.027016+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:39.027016+0000', 'last_peered': '2024-09-11T01:24:39.027016+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:39.027016+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 356352, 'num_objects': 87, 'num_object_clones': 0, 'num_object_copies': 174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 87, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20151, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 990, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3046}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20016", 'reported_seq': 20190, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:41.301827+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:41.301827+0000', 'last_peered': '2024-09-11T01:24:41.301827+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:41.301827+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 118808, 'num_objects': 30, 'num_object_clones': 0, 'num_object_copies': 60, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 30, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18077, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20104", 'reported_seq': 20376, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:38.997388+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:38.997388+0000', 'last_peered': '2024-09-11T01:24:38.997388+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:38.997388+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 422371, 'num_objects': 104, 'num_object_clones': 0, 'num_object_copies': 208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 104, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16592, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3945}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20699", 'reported_seq': 20992, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:38.957013+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:38.957013+0000', 'last_peered': '2024-09-11T01:24:38.957013+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:38.957013+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 463814, 'num_objects': 115, 'num_object_clones': 0, 'num_object_copies': 230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 115, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17452, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4091}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:43.175 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20489", 'reported_seq': 20774, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:39.017024+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:39.017024+0000', 'last_peered': '2024-09-11T01:24:39.017024+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:39.017024+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10400", 'ondisk_log_start': "41'10400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 430080, 'num_objects': 105, 'num_object_clones': 0, 'num_object_copies': 210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16853, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3925}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.176 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:43.176 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20538", 'reported_seq': 20698, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:41.336409+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:41.336409+0000', 'last_peered': '2024-09-11T01:24:41.336409+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:41.336409+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 98304, 'num_objects': 24, 'num_object_clones': 0, 'num_object_copies': 48, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 24, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18175, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4123}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:43.176 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:43.176 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20537", 'reported_seq': 20827, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:39.025555+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:39.025555+0000', 'last_peered': '2024-09-11T01:24:39.025555+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:39.025555+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 446464, 'num_objects': 109, 'num_object_clones': 0, 'num_object_copies': 218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16930, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.176 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:43.176 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20495", 'reported_seq': 20800, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:38.980770+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:38.980770+0000', 'last_peered': '2024-09-11T01:24:38.980770+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:38.980770+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10400", 'ondisk_log_start': "41'10400", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 430080, 'num_objects': 105, 'num_object_clones': 0, 'num_object_copies': 210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17030, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3960}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.177 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:43.177 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20044", 'reported_seq': 20289, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:39.016658+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:39.016658+0000', 'last_peered': '2024-09-11T01:24:39.016658+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:39.016658+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 385990, 'num_objects': 96, 'num_object_clones': 0, 'num_object_copies': 192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 96, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16701, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4006}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:43.177 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:43.567 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:43.567 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35825/2180 objects degraded (1643.349%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":1090,"data_bytes":4416265,"bytes_used":1515356160,"bytes_avail":288394936320,"bytes_total":289910292480,"degraded_objects":35825,"degraded_total":2180,"degraded_ratio":16.433486238532112,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":478},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:43.580 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:43.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:43.930 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:43.930 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:43.945 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T01:24:42.656178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4416265,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":284155,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13810,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160793,"ondisk_log_size":160793,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479840,"kb_used_data":207524,"kb_used_omap":12,"kb_used_meta":1272243,"kb_avail":281635680,"statfs":{"total":289910292480,"available":288394936320,"internally_reserved":0,"allocated":212504576,"data_stored":211587377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1302776839},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,13,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23519232,"num_objects":-5742,"num_object_clones":0,"num_object_copies":-11484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005248"},"pg_stats":[{"pgid":"2.f","version":"45'20151","reported_seq":22211,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.027016+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:39.027016+0000","last_peered":"2024-09-11T01:24:39.027016+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:39.027016+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":356352,"num_objects":87,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20151,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":990,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3046}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20493","reported_seq":20579,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.319148+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:41.319148+0000","last_peered":"2024-09-11T01:24:41.319148+0000","last_clean":"2024-09-11T01:24:41.319148+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:41.319148+0000","last_undegraded":"2024-09-11T01:24:41.319148+0000","last_fullsized":"2024-09-11T01:24:41.319148+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":119750,"num_objects":31,"num_object_clones":0,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18591,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20016","reported_seq":20190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:41.301827+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:41.301827+0000","last_peered":"2024-09-11T01:24:41.301827+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:41.301827+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118808,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18077,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20326","reported_seq":20377,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.348070+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:41.348070+0000","last_peered":"2024-09-11T01:24:41.348070+0000","last_clean":"2024-09-11T01:24:41.348070+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:41.348070+0000","last_undegraded":"2024-09-11T01:24:41.348070+0000","last_fullsized":"2024-09-11T01:24:41.348070+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20326,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20180","reported_seq":20306,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:39.055400+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:39.055400+0000","last_peered":"2024-09-11T01:24:39.055400+0000","last_clean":"2024-09-11T01:24:39.055400+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:39.055400+0000","last_undegraded":"2024-09-11T01:24:39.055400+0000","last_fullsized":"2024-09-11T01:24:39.055400+0000","mapping_epoch":31,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397795,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16978,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20104","reported_seq":20376,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.997388+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:38.997388+0000","last_peered":"2024-09-11T01:24:38.997388+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:38.997388+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":422371,"num_objects":104,"num_object_clones":0,"num_object_copies":208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16592,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3945},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20699","reported_seq":20992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.957013+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:38.957013+0000","last_peered":"2024-09-11T01:24:38.957013+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:38.957013+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463814,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17452,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4091},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20309","reported_seq":20434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:38.992921+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:38.992921+0000","last_peered":"2024-09-11T01:24:38.992921+0000","last_clean":"2024-09-11T01:24:38.992921+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:38.992921+0000","last_undegraded":"2024-09-11T01:24:38.992921+0000","last_fullsized":"2024-09-11T01:24:38.992921+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471040,"num_objects":115,"num_object_clones":0,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17262,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20489","reported_seq":20774,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.017024+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:39.017024+0000","last_peered":"2024-09-11T01:24:39.017024+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:39.017024+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430080,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3925}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20108","reported_seq":28271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:41.197743+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:41.197743+0000","last_peered":"2024-09-11T01:24:41.197743+0000","last_clean":"2024-09-11T01:24:41.197743+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:41.197743+0000","last_undegraded":"2024-09-11T01:24:41.197743+0000","last_fullsized":"2024-09-11T01:24:41.197743+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127459,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17877,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20538","reported_seq":20698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:41.336409+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:41.336409+0000","last_peered":"2024-09-11T01:24:41.336409+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:41.336409+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18175,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4123}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20537","reported_seq":20827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.025555+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:39.025555+0000","last_peered":"2024-09-11T01:24:39.025555+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:39.025555+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446464,"num_objects":109,"num_object_clones":0,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16930,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20495","reported_seq":20800,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.980770+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:38.980770+0000","last_peered":"2024-09-11T01:24:38.980770+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:38.980770+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":430080,"num_objects":105,"num_object_clones":0,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17030,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3960}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20044","reported_seq":20289,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:39.016658+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:39.016658+0000","last_peered":"2024-09-11T01:24:39.016658+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:39.016658+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385990,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16701,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4006},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4416246,"num_objects":1088,"num_object_clones":0,"num_object_copies":2176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1088,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":284153,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13809,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":209403904,"data_stored":209255312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160790,"ondisk_log_size":160790,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473272,"kb_used_data":134760,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93898568,"statfs":{"total":96636764160,"available":96152133632,"internally_reserved":0,"allocated":137994240,"data_stored":137701911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":346615470},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790065,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480536,"kb_used_data":1792,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93891304,"statfs":{"total":96636764160,"available":96144695296,"internally_reserved":0,"allocated":1835008,"data_stored":1502472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526032,"kb_used_data":70972,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93845808,"statfs":{"total":96636764160,"available":96098107392,"internally_reserved":0,"allocated":72675328,"data_stored":72382994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":465956522},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":71979008,"data_stored":71942878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":137342976,"data_stored":137302774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:46.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:47.273 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:47.273 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:47.287 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-11T01:24:46.656910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14070,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465040,"kb_used_data":201048,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281650480,"statfs":{"total":289910292480,"available":288410091520,"internally_reserved":0,"allocated":205873152,"data_stored":204960025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,7,9,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18255896,"num_objects":-4458,"num_object_clones":0,"num_object_copies":-8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4458,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004205"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":22818,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:44.055312+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:44.055312+0000","last_peered":"2024-09-11T01:24:44.055312+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:44.055312+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.287870+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.287870+0000","last_peered":"2024-09-11T01:24:42.287870+0000","last_clean":"2024-09-11T01:24:42.287870+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.287870+0000","last_undegraded":"2024-09-11T01:24:42.287870+0000","last_fullsized":"2024-09-11T01:24:42.287870+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20905,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.313642+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.313642+0000","last_peered":"2024-09-11T01:24:42.313642+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.313642+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14069,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790065,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480536,"kb_used_data":1792,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93891304,"statfs":{"total":96636764160,"available":96144695296,"internally_reserved":0,"allocated":1835008,"data_stored":1502472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:47.289 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:47.600 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:47.601 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:47.613 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-11T01:24:46.656910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14070,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465040,"kb_used_data":201048,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281650480,"statfs":{"total":289910292480,"available":288410091520,"internally_reserved":0,"allocated":205873152,"data_stored":204960025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,7,9,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18255896,"num_objects":-4458,"num_object_clones":0,"num_object_copies":-8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4458,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004205"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":22818,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:44.055312+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:44.055312+0000","last_peered":"2024-09-11T01:24:44.055312+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:44.055312+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.287870+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.287870+0000","last_peered":"2024-09-11T01:24:42.287870+0000","last_clean":"2024-09-11T01:24:42.287870+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.287870+0000","last_undegraded":"2024-09-11T01:24:42.287870+0000","last_fullsized":"2024-09-11T01:24:42.287870+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20905,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.313642+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.313642+0000","last_peered":"2024-09-11T01:24:42.313642+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.313642+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14069,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790065,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480536,"kb_used_data":1792,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93891304,"statfs":{"total":96636764160,"available":96144695296,"internally_reserved":0,"allocated":1835008,"data_stored":1502472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:47.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'20238", 'reported_seq': 22818, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:44.055312+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:44.055312+0000', 'last_peered': '2024-09-11T01:24:44.055312+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:44.055312+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2873, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20238, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1250, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20220, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.322042+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:42.322042+0000', 'last_peered': '2024-09-11T01:24:42.322042+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:42.322042+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20479, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.370854+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:42.370854+0000', 'last_peered': '2024-09-11T01:24:42.370854+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:42.370854+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21105, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.226774+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:42.226774+0000', 'last_peered': '2024-09-11T01:24:42.226774+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:42.226774+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.349937+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:42.349937+0000', 'last_peered': '2024-09-11T01:24:42.349937+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:42.349937+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 20722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.319076+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:42.319076+0000', 'last_peered': '2024-09-11T01:24:42.319076+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:42.319076+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4147}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20936, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.271040+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:42.271040+0000', 'last_peered': '2024-09-11T01:24:42.271040+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:42.271040+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20905, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.313642+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:42.313642+0000', 'last_peered': '2024-09-11T01:24:42.313642+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:42.313642+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:47.616 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20383, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350793+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:42.350793+0000', 'last_peered': '2024-09-11T01:24:42.350793+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:42.350793+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:47.616 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:48.076 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:48.076 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35565/24 objects degraded (148187.500%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":688,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1500200960,"bytes_avail":288410091520,"bytes_total":289910292480,"degraded_objects":35565,"degraded_total":24,"degraded_ratio":1481.875,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":346,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":371},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:48.092 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:48.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:48.416 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:48.416 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:48.432 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-11T01:24:46.656910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14070,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465040,"kb_used_data":201048,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281650480,"statfs":{"total":289910292480,"available":288410091520,"internally_reserved":0,"allocated":205873152,"data_stored":204960025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,7,9,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18255896,"num_objects":-4458,"num_object_clones":0,"num_object_copies":-8916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-346,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4458,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":346,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004205"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":22818,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:44.055312+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:44.055312+0000","last_peered":"2024-09-11T01:24:44.055312+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:44.055312+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.287870+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.287870+0000","last_peered":"2024-09-11T01:24:42.287870+0000","last_clean":"2024-09-11T01:24:42.287870+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.287870+0000","last_undegraded":"2024-09-11T01:24:42.287870+0000","last_fullsized":"2024-09-11T01:24:42.287870+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20905,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.313642+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.313642+0000","last_peered":"2024-09-11T01:24:42.313642+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.313642+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35565,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14069,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790065,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480536,"kb_used_data":1792,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93891304,"statfs":{"total":96636764160,"available":96144695296,"internally_reserved":0,"allocated":1835008,"data_stored":1502472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:51.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:51.746 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:51.747 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:51.760 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-11T01:24:50.657667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14697,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467852,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647668,"statfs":{"total":289910292480,"available":288407212032,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14041112,"num_objects":-3429,"num_object_clones":0,"num_object_copies":-6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003102"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":24072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:49.053581+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:49.053581+0000","last_peered":"2024-09-11T01:24:49.053581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:49.053581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2246}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.287870+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.287870+0000","last_peered":"2024-09-11T01:24:42.287870+0000","last_clean":"2024-09-11T01:24:42.287870+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.287870+0000","last_undegraded":"2024-09-11T01:24:42.287870+0000","last_fullsized":"2024-09-11T01:24:42.287870+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14696,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790066,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483348,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888492,"statfs":{"total":96636764160,"available":96141815808,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:51.762 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:52.077 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:52.077 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:52.089 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-11T01:24:50.657667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14697,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467852,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647668,"statfs":{"total":289910292480,"available":288407212032,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14041112,"num_objects":-3429,"num_object_clones":0,"num_object_copies":-6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":973,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003102"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":24072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:49.053581+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:49.053581+0000","last_peered":"2024-09-11T01:24:49.053581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:49.053581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2246}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841875+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841875+0000","last_peered":"2024-09-11T01:23:46.841875+0000","last_clean":"2024-09-11T01:23:46.841875+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841875+0000","last_undegraded":"2024-09-11T01:23:46.841875+0000","last_fullsized":"2024-09-11T01:23:46.841875+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.287870+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.287870+0000","last_peered":"2024-09-11T01:24:42.287870+0000","last_clean":"2024-09-11T01:24:42.287870+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.287870+0000","last_undegraded":"2024-09-11T01:24:42.287870+0000","last_fullsized":"2024-09-11T01:24:42.287870+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842738+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842738+0000","last_peered":"2024-09-11T01:23:46.842738+0000","last_clean":"2024-09-11T01:23:46.842738+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842738+0000","last_undegraded":"2024-09-11T01:23:46.842738+0000","last_fullsized":"2024-09-11T01:23:46.842738+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14696,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790066,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483348,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888492,"statfs":{"total":96636764160,"available":96141815808,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'20238", 'reported_seq': 24072, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:49.053581+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:49.053581+0000', 'last_peered': '2024-09-11T01:24:49.053581+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:49.053581+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2246, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20238, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1877, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2246}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20220, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.322042+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:42.322042+0000', 'last_peered': '2024-09-11T01:24:42.322042+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:42.322042+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20479, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.370854+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:42.370854+0000', 'last_peered': '2024-09-11T01:24:42.370854+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:42.370854+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:52.091 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21105, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.226774+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:42.226774+0000', 'last_peered': '2024-09-11T01:24:42.226774+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:42.226774+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.349937+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:42.349937+0000', 'last_peered': '2024-09-11T01:24:42.349937+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:42.349937+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 20722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.319076+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:42.319076+0000', 'last_peered': '2024-09-11T01:24:42.319076+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:42.319076+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4147}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20936, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.271040+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:42.271040+0000', 'last_peered': '2024-09-11T01:24:42.271040+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:42.271040+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350364+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:42.350364+0000', 'last_peered': '2024-09-11T01:24:42.350364+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:42.350364+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:52.092 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20383, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350793+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:42.350793+0000', 'last_peered': '2024-09-11T01:24:42.350793+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:42.350793+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:52.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:52.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:52.476 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34938/24 objects degraded (145575.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":692,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1503080448,"bytes_avail":288407212032,"bytes_total":289910292480,"degraded_objects":34938,"degraded_total":24,"degraded_ratio":1455.75,"recovering_objects_per_sec":81,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":973,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":285},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:52.489 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:52.489 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:52.810 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:52.810 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:52.824 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-11T01:24:52.658066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14697,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476172,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639348,"statfs":{"total":289910292480,"available":288398692352,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8482840,"num_objects":-2072,"num_object_clones":0,"num_object_copies":-4144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2072,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":887,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002590"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":24072,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:49.053581+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:49.053581+0000","last_peered":"2024-09-11T01:24:49.053581+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:49.053581+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1877,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2246}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14696,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790067,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491668,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880172,"statfs":{"total":96636764160,"available":96133296128,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:55.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:56.168 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:56.169 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:56.183 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-11T01:24:54.658465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15460,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476172,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639348,"statfs":{"total":289910292480,"available":288398692352,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4411416,"num_objects":-1078,"num_object_clones":0,"num_object_copies":-2156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1650,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002286"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":25598,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:54.059627+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:54.059627+0000","last_peered":"2024-09-11T01:24:54.059627+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:54.059627+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2640,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1483}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15459,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790067,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491668,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880172,"statfs":{"total":96636764160,"available":96133296128,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:56.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:56.510 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:56.510 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:56.524 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-11T01:24:54.658465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15460,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476172,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639348,"statfs":{"total":289910292480,"available":288398692352,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4411416,"num_objects":-1078,"num_object_clones":0,"num_object_copies":-2156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1078,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1650,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002286"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":25598,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:54.059627+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:54.059627+0000","last_peered":"2024-09-11T01:24:54.059627+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:54.059627+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2640,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1483}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15459,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790067,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491668,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880172,"statfs":{"total":96636764160,"available":96133296128,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:56.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:24:56.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'20238", 'reported_seq': 25598, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:54.059627+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:54.059627+0000', 'last_peered': '2024-09-11T01:24:54.059627+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:54.059627+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1483, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20238, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2640, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1483}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:24:56.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20220, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.322042+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:42.322042+0000', 'last_peered': '2024-09-11T01:24:42.322042+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:42.322042+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20479, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.370854+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:42.370854+0000', 'last_peered': '2024-09-11T01:24:42.370854+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:42.370854+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21105, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.226774+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:42.226774+0000', 'last_peered': '2024-09-11T01:24:42.226774+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:42.226774+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.349937+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:42.349937+0000', 'last_peered': '2024-09-11T01:24:42.349937+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:42.349937+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 20722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.319076+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:42.319076+0000', 'last_peered': '2024-09-11T01:24:42.319076+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:42.319076+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4147}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:24:56.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20936, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.271040+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:42.271040+0000', 'last_peered': '2024-09-11T01:24:42.271040+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:42.271040+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:24:56.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350364+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:42.350364+0000', 'last_peered': '2024-09-11T01:24:42.350364+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:42.350364+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.527 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:56.527 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20383, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350793+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:42.350793+0000', 'last_peered': '2024-09-11T01:24:42.350793+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:42.350793+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:56.527 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:56.982 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:56.982 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34175/24 objects degraded (142395.833%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":697,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1511600128,"bytes_avail":288398692352,"bytes_total":289910292480,"degraded_objects":34175,"degraded_total":24,"degraded_ratio":1423.958333333333,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1390,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":12},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:56.996 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:56.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:57.308 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:24:57.308 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:24:57.324 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T01:24:56.658823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15460,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476172,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639348,"statfs":{"total":289910292480,"available":288398692352,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-610328,"num_objects":-150,"num_object_clones":0,"num_object_copies":-300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-150,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":150,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1390,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002259"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":25598,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:54.059627+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:54.059627+0000","last_peered":"2024-09-11T01:24:54.059627+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:54.059627+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2640,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1483}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15459,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790067,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491668,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880172,"statfs":{"total":96636764160,"available":96133296128,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:00.328 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:00.637 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:00.638 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:00.649 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-11T01:24:58.659170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15460,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463692,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281651828,"statfs":{"total":289910292480,"available":288411471872,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1390,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002260"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":25598,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:54.059627+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:54.059627+0000","last_peered":"2024-09-11T01:24:54.059627+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:54.059627+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2640,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1483}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15459,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790068,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:00.651 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:00.962 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:00.963 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:00.976 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-11T01:25:00.659492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16482,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463692,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281651828,"statfs":{"total":289910292480,"available":288411471872,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002162"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":27642,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:59.058186+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:59.058186+0000","last_peered":"2024-09-11T01:24:59.058186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:59.058186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-461}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16481,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790068,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:00.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:25:00.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'20238", 'reported_seq': 27642, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:59.058186+0000', 'last_change': '2024-09-11T01:23:45.349686+0000', 'last_active': '2024-09-11T01:24:59.058186+0000', 'last_peered': '2024-09-11T01:24:59.058186+0000', 'last_clean': '2024-09-11T01:15:17.959089+0000', 'last_became_active': '2024-09-11T01:15:20.485226+0000', 'last_became_peered': '2024-09-11T01:15:20.485226+0000', 'last_unstale': '2024-09-11T01:24:59.058186+0000', 'last_undegraded': '2024-09-11T01:15:20.065206+0000', 'last_fullsized': '2024-09-11T01:15:19.985210+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 461, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20238, 'num_write_kb': 40476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3662, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -461}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:00.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20220, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.322042+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:42.322042+0000', 'last_peered': '2024-09-11T01:24:42.322042+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:42.322042+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:00.977 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:00.977 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20479, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.370854+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:24:42.370854+0000', 'last_peered': '2024-09-11T01:24:42.370854+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:24:42.370854+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21105, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.226774+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:24:42.226774+0000', 'last_peered': '2024-09-11T01:24:42.226774+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:24:42.226774+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20879, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.349937+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:24:42.349937+0000', 'last_peered': '2024-09-11T01:24:42.349937+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:24:42.349937+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 20722, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.319076+0000', 'last_change': '2024-09-11T01:15:20.522607+0000', 'last_active': '2024-09-11T01:24:42.319076+0000', 'last_peered': '2024-09-11T01:24:42.319076+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:24:42.319076+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 30, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4147}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20936, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.271040+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:24:42.271040+0000', 'last_peered': '2024-09-11T01:24:42.271040+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:24:42.271040+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350364+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:24:42.350364+0000', 'last_peered': '2024-09-11T01:24:42.350364+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:24:42.350364+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:00.978 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20383, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.350793+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:24:42.350793+0000', 'last_peered': '2024-09-11T01:24:42.350793+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:24:42.350793+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:00.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:01.366 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:01.366 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33153/24 objects degraded (138137.500%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":701,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1498820608,"bytes_avail":288411471872,"bytes_total":289910292480,"degraded_objects":33153,"degraded_total":24,"degraded_ratio":1381.375,"recovering_objects_per_sec":200,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2412,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:01.380 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:01.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:01.700 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:01.701 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:01.715 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-09-11T01:25:00.659492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16482,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463692,"kb_used_data":199708,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281651828,"statfs":{"total":289910292480,"available":288411471872,"internally_reserved":0,"allocated":204500992,"data_stored":203591937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002162"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":27642,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:59.058186+0000","last_change":"2024-09-11T01:23:45.349686+0000","last_active":"2024-09-11T01:24:59.058186+0000","last_peered":"2024-09-11T01:24:59.058186+0000","last_clean":"2024-09-11T01:15:17.959089+0000","last_became_active":"2024-09-11T01:15:20.485226+0000","last_became_peered":"2024-09-11T01:15:20.485226+0000","last_unstale":"2024-09-11T01:24:59.058186+0000","last_undegraded":"2024-09-11T01:15:20.065206+0000","last_fullsized":"2024-09-11T01:15:19.985210+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3662,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-461}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20404,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.353821+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:24:42.353821+0000","last_peered":"2024-09-11T01:24:42.353821+0000","last_clean":"2024-09-11T01:24:42.353821+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:24:42.353821+0000","last_undegraded":"2024-09-11T01:24:42.353821+0000","last_fullsized":"2024-09-11T01:24:42.353821+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.370854+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:24:42.370854+0000","last_peered":"2024-09-11T01:24:42.370854+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:24:42.370854+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21105,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.226774+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:24:42.226774+0000","last_peered":"2024-09-11T01:24:42.226774+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:24:42.226774+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20549,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.244565+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:24:42.244565+0000","last_peered":"2024-09-11T01:24:42.244565+0000","last_clean":"2024-09-11T01:24:42.244565+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:24:42.244565+0000","last_undegraded":"2024-09-11T01:24:42.244565+0000","last_fullsized":"2024-09-11T01:24:42.244565+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.839823+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:23:46.839823+0000","last_peered":"2024-09-11T01:23:46.839823+0000","last_clean":"2024-09-11T01:23:46.839823+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:23:46.839823+0000","last_undegraded":"2024-09-11T01:23:46.839823+0000","last_fullsized":"2024-09-11T01:23:46.839823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.285855+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:24:42.285855+0000","last_peered":"2024-09-11T01:24:42.285855+0000","last_clean":"2024-09-11T01:24:42.285855+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:24:42.285855+0000","last_undegraded":"2024-09-11T01:24:42.285855+0000","last_fullsized":"2024-09-11T01:24:42.285855+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20879,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.349937+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:24:42.349937+0000","last_peered":"2024-09-11T01:24:42.349937+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:24:42.349937+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":104,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843425+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:23:46.843425+0000","last_peered":"2024-09-11T01:23:46.843425+0000","last_clean":"2024-09-11T01:23:46.843425+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:23:46.843425+0000","last_undegraded":"2024-09-11T01:23:46.843425+0000","last_fullsized":"2024-09-11T01:23:46.843425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843044+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:23:46.843044+0000","last_peered":"2024-09-11T01:23:46.843044+0000","last_clean":"2024-09-11T01:23:46.843044+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:23:46.843044+0000","last_undegraded":"2024-09-11T01:23:46.843044+0000","last_fullsized":"2024-09-11T01:23:46.843044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20722,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.319076+0000","last_change":"2024-09-11T01:15:20.522607+0000","last_active":"2024-09-11T01:24:42.319076+0000","last_peered":"2024-09-11T01:24:42.319076+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:24:42.319076+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4147}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28828,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.304975+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:24:42.304975+0000","last_peered":"2024-09-11T01:24:42.304975+0000","last_clean":"2024-09-11T01:24:42.304975+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:24:42.304975+0000","last_undegraded":"2024-09-11T01:24:42.304975+0000","last_fullsized":"2024-09-11T01:24:42.304975+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843221+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:23:46.843221+0000","last_peered":"2024-09-11T01:23:46.843221+0000","last_clean":"2024-09-11T01:23:46.843221+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:23:46.843221+0000","last_undegraded":"2024-09-11T01:23:46.843221+0000","last_fullsized":"2024-09-11T01:23:46.843221+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20936,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.271040+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:24:42.271040+0000","last_peered":"2024-09-11T01:24:42.271040+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:24:42.271040+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350364+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:24:42.350364+0000","last_peered":"2024-09-11T01:24:42.350364+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:24:42.350364+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843469+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:23:46.843469+0000","last_peered":"2024-09-11T01:23:46.843469+0000","last_clean":"2024-09-11T01:23:46.843469+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:23:46.843469+0000","last_undegraded":"2024-09-11T01:23:46.843469+0000","last_fullsized":"2024-09-11T01:23:46.843469+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.843149+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:23:46.843149+0000","last_peered":"2024-09-11T01:23:46.843149+0000","last_clean":"2024-09-11T01:23:46.843149+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:23:46.843149+0000","last_undegraded":"2024-09-11T01:23:46.843149+0000","last_fullsized":"2024-09-11T01:23:46.843149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.350793+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:24:42.350793+0000","last_peered":"2024-09-11T01:24:42.350793+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:24:42.350793+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16481,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203362304,"data_stored":203217784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790068,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":462848,"data_stored":134384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508308,"kb_used_data":65732,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863532,"statfs":{"total":96636764160,"available":96116256768,"internally_reserved":0,"allocated":67309568,"data_stored":67017234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":9660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:04.718 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:05.033 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:05.033 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:05.044 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-11T01:25:04.660261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16997,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476184,"kb_used_data":199716,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639336,"statfs":{"total":289910292480,"available":288398680064,"internally_reserved":0,"allocated":204509184,"data_stored":203596804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001438"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830486+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:01.830486+0000","last_peered":"2024-09-11T01:25:01.830486+0000","last_clean":"2024-09-11T01:25:01.830486+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:01.830486+0000","last_undegraded":"2024-09-11T01:25:01.830486+0000","last_fullsized":"2024-09-11T01:25:01.830486+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28600,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830195+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:01.830195+0000","last_peered":"2024-09-11T01:25:01.830195+0000","last_clean":"2024-09-11T01:25:01.830195+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:01.830195+0000","last_undegraded":"2024-09-11T01:25:01.830195+0000","last_fullsized":"2024-09-11T01:25:01.830195+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830099+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:01.830099+0000","last_peered":"2024-09-11T01:25:01.830099+0000","last_clean":"2024-09-11T01:25:01.830099+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:01.830099+0000","last_undegraded":"2024-09-11T01:25:01.830099+0000","last_fullsized":"2024-09-11T01:25:01.830099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20831,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:01.364012+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:01.364012+0000","last_peered":"2024-09-11T01:25:01.364012+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:01.364012+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830124+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:01.830124+0000","last_peered":"2024-09-11T01:25:01.830124+0000","last_clean":"2024-09-11T01:25:01.830124+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:01.830124+0000","last_undegraded":"2024-09-11T01:25:01.830124+0000","last_fullsized":"2024-09-11T01:25:01.830124+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830285+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:01.830285+0000","last_peered":"2024-09-11T01:25:01.830285+0000","last_clean":"2024-09-11T01:25:01.830285+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:01.830285+0000","last_undegraded":"2024-09-11T01:25:01.830285+0000","last_fullsized":"2024-09-11T01:25:01.830285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839621+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.839621+0000","last_peered":"2024-09-11T01:25:02.839621+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.839621+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830395+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:01.830395+0000","last_peered":"2024-09-11T01:25:01.830395+0000","last_clean":"2024-09-11T01:25:01.830395+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:01.830395+0000","last_undegraded":"2024-09-11T01:25:01.830395+0000","last_fullsized":"2024-09-11T01:25:01.830395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830012+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:01.830012+0000","last_peered":"2024-09-11T01:25:01.830012+0000","last_clean":"2024-09-11T01:25:01.830012+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:01.830012+0000","last_undegraded":"2024-09-11T01:25:01.830012+0000","last_fullsized":"2024-09-11T01:25:01.830012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16996,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790069,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":454656,"data_stored":133418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:05.046 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:05.359 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:05.360 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:05.372 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-11T01:25:04.660261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16997,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476184,"kb_used_data":199716,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639336,"statfs":{"total":289910292480,"available":288398680064,"internally_reserved":0,"allocated":204509184,"data_stored":203596804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001438"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830486+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:01.830486+0000","last_peered":"2024-09-11T01:25:01.830486+0000","last_clean":"2024-09-11T01:25:01.830486+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:01.830486+0000","last_undegraded":"2024-09-11T01:25:01.830486+0000","last_fullsized":"2024-09-11T01:25:01.830486+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28600,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830195+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:01.830195+0000","last_peered":"2024-09-11T01:25:01.830195+0000","last_clean":"2024-09-11T01:25:01.830195+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:01.830195+0000","last_undegraded":"2024-09-11T01:25:01.830195+0000","last_fullsized":"2024-09-11T01:25:01.830195+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830099+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:01.830099+0000","last_peered":"2024-09-11T01:25:01.830099+0000","last_clean":"2024-09-11T01:25:01.830099+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:01.830099+0000","last_undegraded":"2024-09-11T01:25:01.830099+0000","last_fullsized":"2024-09-11T01:25:01.830099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20831,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:01.364012+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:01.364012+0000","last_peered":"2024-09-11T01:25:01.364012+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:01.364012+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830124+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:01.830124+0000","last_peered":"2024-09-11T01:25:01.830124+0000","last_clean":"2024-09-11T01:25:01.830124+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:01.830124+0000","last_undegraded":"2024-09-11T01:25:01.830124+0000","last_fullsized":"2024-09-11T01:25:01.830124+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830285+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:01.830285+0000","last_peered":"2024-09-11T01:25:01.830285+0000","last_clean":"2024-09-11T01:25:01.830285+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:01.830285+0000","last_undegraded":"2024-09-11T01:25:01.830285+0000","last_fullsized":"2024-09-11T01:25:01.830285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839621+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.839621+0000","last_peered":"2024-09-11T01:25:02.839621+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.839621+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830395+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:01.830395+0000","last_peered":"2024-09-11T01:25:01.830395+0000","last_clean":"2024-09-11T01:25:01.830395+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:01.830395+0000","last_undegraded":"2024-09-11T01:25:01.830395+0000","last_fullsized":"2024-09-11T01:25:01.830395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830012+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:01.830012+0000","last_peered":"2024-09-11T01:25:01.830012+0000","last_clean":"2024-09-11T01:25:01.830012+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:01.830012+0000","last_undegraded":"2024-09-11T01:25:01.830012+0000","last_fullsized":"2024-09-11T01:25:01.830012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16996,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790069,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":454656,"data_stored":133418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20220, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:42.322042+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:24:42.322042+0000', 'last_peered': '2024-09-11T01:24:42.322042+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:24:42.322042+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20483, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839053+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:02.839053+0000', 'last_peered': '2024-09-11T01:25:02.839053+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:02.839053+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21109, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.838966+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:02.838966+0000', 'last_peered': '2024-09-11T01:25:02.838966+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:02.838966+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:05.374 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839073+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:02.839073+0000', 'last_peered': '2024-09-11T01:25:02.839073+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:02.839073+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 20831, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:01.364012+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:01.364012+0000', 'last_peered': '2024-09-11T01:25:01.364012+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:01.364012+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 84, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4095}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20940, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839621+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:02.839621+0000', 'last_peered': '2024-09-11T01:25:02.839621+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:02.839621+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839646+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:02.839646+0000', 'last_peered': '2024-09-11T01:25:02.839646+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:02.839646+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:05.375 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839253+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:25:02.839253+0000', 'last_peered': '2024-09-11T01:25:02.839253+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:02.839253+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:05.376 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:05.751 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32638/24 objects degraded (135991.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":706,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1511612416,"bytes_avail":288398680064,"bytes_total":289910292480,"degraded_objects":32638,"degraded_total":24,"degraded_ratio":1359.916666666667,"recovering_objects_per_sec":192,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:05.764 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:05.764 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:06.093 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:06.094 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:06.107 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-11T01:25:04.660261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16997,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476184,"kb_used_data":199716,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639336,"statfs":{"total":289910292480,"available":288398680064,"internally_reserved":0,"allocated":204509184,"data_stored":203596804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1537,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001438"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20608,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.336459+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:24:42.336459+0000","last_peered":"2024-09-11T01:24:42.336459+0000","last_clean":"2024-09-11T01:24:42.336459+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:24:42.336459+0000","last_undegraded":"2024-09-11T01:24:42.336459+0000","last_fullsized":"2024-09-11T01:24:42.336459+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20220,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:42.322042+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:24:42.322042+0000","last_peered":"2024-09-11T01:24:42.322042+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:24:42.322042+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.280567+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:24:42.280567+0000","last_peered":"2024-09-11T01:24:42.280567+0000","last_clean":"2024-09-11T01:24:42.280567+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:24:42.280567+0000","last_undegraded":"2024-09-11T01:24:42.280567+0000","last_fullsized":"2024-09-11T01:24:42.280567+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830486+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:01.830486+0000","last_peered":"2024-09-11T01:25:01.830486+0000","last_clean":"2024-09-11T01:25:01.830486+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:01.830486+0000","last_undegraded":"2024-09-11T01:25:01.830486+0000","last_fullsized":"2024-09-11T01:25:01.830486+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28600,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830195+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:01.830195+0000","last_peered":"2024-09-11T01:25:01.830195+0000","last_clean":"2024-09-11T01:25:01.830195+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:01.830195+0000","last_undegraded":"2024-09-11T01:25:01.830195+0000","last_fullsized":"2024-09-11T01:25:01.830195+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.841917+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:23:46.841917+0000","last_peered":"2024-09-11T01:23:46.841917+0000","last_clean":"2024-09-11T01:23:46.841917+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:23:46.841917+0000","last_undegraded":"2024-09-11T01:23:46.841917+0000","last_fullsized":"2024-09-11T01:23:46.841917+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28303,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:24:42.301341+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:24:42.301341+0000","last_peered":"2024-09-11T01:24:42.301341+0000","last_clean":"2024-09-11T01:24:42.301341+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:24:42.301341+0000","last_undegraded":"2024-09-11T01:24:42.301341+0000","last_fullsized":"2024-09-11T01:24:42.301341+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":92,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830099+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:01.830099+0000","last_peered":"2024-09-11T01:25:01.830099+0000","last_clean":"2024-09-11T01:25:01.830099+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:01.830099+0000","last_undegraded":"2024-09-11T01:25:01.830099+0000","last_fullsized":"2024-09-11T01:25:01.830099+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":20831,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:01.364012+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:01.364012+0000","last_peered":"2024-09-11T01:25:01.364012+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:01.364012+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28830,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830124+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:01.830124+0000","last_peered":"2024-09-11T01:25:01.830124+0000","last_clean":"2024-09-11T01:25:01.830124+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:01.830124+0000","last_undegraded":"2024-09-11T01:25:01.830124+0000","last_fullsized":"2024-09-11T01:25:01.830124+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830285+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:01.830285+0000","last_peered":"2024-09-11T01:25:01.830285+0000","last_clean":"2024-09-11T01:25:01.830285+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:01.830285+0000","last_undegraded":"2024-09-11T01:25:01.830285+0000","last_fullsized":"2024-09-11T01:25:01.830285+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839621+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.839621+0000","last_peered":"2024-09-11T01:25:02.839621+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.839621+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:23:46.842771+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:23:46.842771+0000","last_peered":"2024-09-11T01:23:46.842771+0000","last_clean":"2024-09-11T01:23:46.842771+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:23:46.842771+0000","last_undegraded":"2024-09-11T01:23:46.842771+0000","last_fullsized":"2024-09-11T01:23:46.842771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830395+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:01.830395+0000","last_peered":"2024-09-11T01:25:01.830395+0000","last_clean":"2024-09-11T01:25:01.830395+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:01.830395+0000","last_undegraded":"2024-09-11T01:25:01.830395+0000","last_fullsized":"2024-09-11T01:25:01.830395+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-11T01:25:01.830012+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:01.830012+0000","last_peered":"2024-09-11T01:25:01.830012+0000","last_clean":"2024-09-11T01:25:01.830012+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:01.830012+0000","last_undegraded":"2024-09-11T01:25:01.830012+0000","last_fullsized":"2024-09-11T01:25:01.830012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32638,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16996,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476196,"kb_used_data":133524,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895644,"statfs":{"total":96636764160,"available":96149139456,"internally_reserved":0,"allocated":136728576,"data_stored":136440319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790069,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":454656,"data_stored":133418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:09.110 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:09.421 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:09.421 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:09.435 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-11T01:25:08.660990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17760,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463744,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281651776,"statfs":{"total":289910292480,"available":288411418624,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001497"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":22361,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:06.362031+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:06.362031+0000","last_peered":"2024-09-11T01:25:06.362031+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:06.362031+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839621+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.839621+0000","last_peered":"2024-09-11T01:25:02.839621+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.839621+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17759,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790070,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479196,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892644,"statfs":{"total":96636764160,"available":96146067456,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:09.436 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:09.743 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:09.744 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:09.756 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-11T01:25:08.660990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17760,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463744,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281651776,"statfs":{"total":289910292480,"available":288411418624,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001497"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":22361,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:06.362031+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:06.362031+0000","last_peered":"2024-09-11T01:25:06.362031+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:06.362031+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839621+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.839621+0000","last_peered":"2024-09-11T01:25:02.839621+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.839621+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17759,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790070,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479196,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892644,"statfs":{"total":96636764160,"available":96146067456,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.841309+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:02.841309+0000', 'last_peered': '2024-09-11T01:25:02.841309+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:02.841309+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20483, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839053+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:02.839053+0000', 'last_peered': '2024-09-11T01:25:02.839053+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:02.839053+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21109, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.838966+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:02.838966+0000', 'last_peered': '2024-09-11T01:25:02.838966+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:02.838966+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:09.758 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839073+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:02.839073+0000', 'last_peered': '2024-09-11T01:25:02.839073+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:02.839073+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 22361, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:06.362031+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:06.362031+0000', 'last_peered': '2024-09-11T01:25:06.362031+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:06.362031+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3332, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 847, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3332}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20940, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839621+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:02.839621+0000', 'last_peered': '2024-09-11T01:25:02.839621+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:02.839621+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839646+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:02.839646+0000', 'last_peered': '2024-09-11T01:25:02.839646+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:02.839646+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:09.759 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839253+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:25:02.839253+0000', 'last_peered': '2024-09-11T01:25:02.839253+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:02.839253+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:09.760 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:10.147 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:10.148 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31875/24 objects degraded (132812.500%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":710,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1498873856,"bytes_avail":288411418624,"bytes_total":289910292480,"degraded_objects":31875,"degraded_total":24,"degraded_ratio":1328.125,"recovering_objects_per_sec":159,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:10.160 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:10.160 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:10.475 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:10.476 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:10.489 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-11T01:25:08.660990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17760,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463744,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1263923,"kb_avail":281651776,"statfs":{"total":289910292480,"available":288411418624,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1294257159},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001497"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":22361,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:06.362031+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:06.362031+0000","last_peered":"2024-09-11T01:25:06.362031+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:06.362031+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":847,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20940,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839621+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.839621+0000","last_peered":"2024-09-11T01:25:02.839621+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.839621+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31875,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17759,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790070,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479196,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892644,"statfs":{"total":96636764160,"available":96146067456,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:13.491 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:13.795 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:13.796 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:13.808 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-11T01:25:12.661740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18365,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001814"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":23571,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:11.363746+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:11.363746+0000","last_peered":"2024-09-11T01:25:11.363746+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:11.363746+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18364,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:13.809 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:14.125 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:14.135 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-11T01:25:12.661740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18365,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001814"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839053+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.839053+0000","last_peered":"2024-09-11T01:25:02.839053+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.839053+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21109,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.838966+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.838966+0000","last_peered":"2024-09-11T01:25:02.838966+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.838966+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":23571,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:11.363746+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:11.363746+0000","last_peered":"2024-09-11T01:25:11.363746+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:11.363746+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18364,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:14.137 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:14.137 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.841309+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:02.841309+0000', 'last_peered': '2024-09-11T01:25:02.841309+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:02.841309+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:14.137 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:14.137 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20483, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839053+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:02.839053+0000', 'last_peered': '2024-09-11T01:25:02.839053+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:02.839053+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21109, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.838966+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:02.838966+0000', 'last_peered': '2024-09-11T01:25:02.838966+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:02.838966+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839073+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:02.839073+0000', 'last_peered': '2024-09-11T01:25:02.839073+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:02.839073+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 23571, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:11.363746+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:11.363746+0000', 'last_peered': '2024-09-11T01:25:11.363746+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:11.363746+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2727, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1452, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2727}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20941, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.840898+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:02.840898+0000', 'last_peered': '2024-09-11T01:25:02.840898+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:02.840898+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:14.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839646+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:02.839646+0000', 'last_peered': '2024-09-11T01:25:02.839646+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:02.839646+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:14.139 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:14.139 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839253+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:25:02.839253+0000', 'last_peered': '2024-09-11T01:25:02.839253+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:02.839253+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:14.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:14.531 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:14.532 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31270/24 objects degraded (130291.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":714,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1503133696,"bytes_avail":288407158784,"bytes_total":289910292480,"degraded_objects":31270,"degraded_total":24,"degraded_ratio":1302.9166666666667,"recovering_objects_per_sec":182,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:14.545 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:14.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:14.884 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:14.885 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:14.897 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-11T01:25:14.662145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18365,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.823441"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":23571,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:11.363746+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:11.363746+0000","last_peered":"2024-09-11T01:25:11.363746+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:11.363746+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1452,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2727}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18364,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:17.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:18.228 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:18.228 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:18.240 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-11T01:25:16.662600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19055,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002338"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":24951,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:16.368709+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:16.368709+0000","last_peered":"2024-09-11T01:25:16.368709+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:16.368709+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:18.241 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:18.546 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:18.546 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:18.558 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-11T01:25:16.662600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19055,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002338"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":24951,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:16.368709+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:16.368709+0000","last_peered":"2024-09-11T01:25:16.368709+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:16.368709+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.841309+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:02.841309+0000', 'last_peered': '2024-09-11T01:25:02.841309+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:02.841309+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20484, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.843770+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:02.843770+0000', 'last_peered': '2024-09-11T01:25:02.843770+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:02.843770+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21110, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839912+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:02.839912+0000', 'last_peered': '2024-09-11T01:25:02.839912+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:02.839912+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:18.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839073+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:02.839073+0000', 'last_peered': '2024-09-11T01:25:02.839073+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:02.839073+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 24951, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:16.368709+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:16.368709+0000', 'last_peered': '2024-09-11T01:25:16.368709+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:16.368709+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2037, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2142, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2037}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20941, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.840898+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:02.840898+0000', 'last_peered': '2024-09-11T01:25:02.840898+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:02.840898+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839646+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:02.839646+0000', 'last_peered': '2024-09-11T01:25:02.839646+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:02.839646+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:18.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839253+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:25:02.839253+0000', 'last_peered': '2024-09-11T01:25:02.839253+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:02.839253+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:18.562 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:18.960 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:18.961 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30580/24 objects degraded (127416.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":719,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1511653376,"bytes_avail":288398639104,"bytes_total":289910292480,"degraded_objects":30580,"degraded_total":24,"degraded_ratio":1274.1666666666667,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1295,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:18.974 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:18.974 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:19.295 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:19.295 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:19.309 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-11T01:25:18.662938+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19055,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476224,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639296,"statfs":{"total":289910292480,"available":288398639104,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002284"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20408,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839145+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839145+0000","last_peered":"2024-09-11T01:25:02.839145+0000","last_clean":"2024-09-11T01:25:02.839145+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839145+0000","last_undegraded":"2024-09-11T01:25:02.839145+0000","last_fullsized":"2024-09-11T01:25:02.839145+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":24951,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:16.368709+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:16.368709+0000","last_peered":"2024-09-11T01:25:16.368709+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:16.368709+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790072,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491676,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880164,"statfs":{"total":96636764160,"available":96133287936,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:22.311 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:22.634 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:22.635 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:22.647 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-11T01:25:20.663405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19055,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476224,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639296,"statfs":{"total":289910292480,"available":288398639104,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002415"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839185+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839185+0000","last_peered":"2024-09-11T01:25:02.839185+0000","last_clean":"2024-09-11T01:25:02.839185+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839185+0000","last_undegraded":"2024-09-11T01:25:02.839185+0000","last_fullsized":"2024-09-11T01:25:02.839185+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842747+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842747+0000","last_peered":"2024-09-11T01:25:02.842747+0000","last_clean":"2024-09-11T01:25:02.842747+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842747+0000","last_undegraded":"2024-09-11T01:25:02.842747+0000","last_fullsized":"2024-09-11T01:25:02.842747+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":24951,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:16.368709+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:16.368709+0000","last_peered":"2024-09-11T01:25:16.368709+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:16.368709+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2037,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2037}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19054,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790072,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491676,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880164,"statfs":{"total":96636764160,"available":96133287936,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:22.649 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:22.967 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:22.968 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:22.981 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-11T01:25:22.663761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19893,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002430"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839185+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839185+0000","last_peered":"2024-09-11T01:25:02.839185+0000","last_clean":"2024-09-11T01:25:02.839185+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839185+0000","last_undegraded":"2024-09-11T01:25:02.839185+0000","last_fullsized":"2024-09-11T01:25:02.839185+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842776+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842776+0000","last_peered":"2024-09-11T01:25:02.842776+0000","last_clean":"2024-09-11T01:25:02.842776+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842776+0000","last_undegraded":"2024-09-11T01:25:02.842776+0000","last_fullsized":"2024-09-11T01:25:02.842776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":26627,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:21.369283+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:21.369283+0000","last_peered":"2024-09-11T01:25:21.369283+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:21.369283+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1199}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19892,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790073,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:22.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:22.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.841309+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:02.841309+0000', 'last_peered': '2024-09-11T01:25:02.841309+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:02.841309+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:22.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:22.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20484, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.843770+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:02.843770+0000', 'last_peered': '2024-09-11T01:25:02.843770+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:02.843770+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:22.982 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:22.982 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21110, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839912+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:02.839912+0000', 'last_peered': '2024-09-11T01:25:02.839912+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:02.839912+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839073+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:02.839073+0000', 'last_peered': '2024-09-11T01:25:02.839073+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:02.839073+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 26627, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:21.369283+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:21.369283+0000', 'last_peered': '2024-09-11T01:25:21.369283+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:21.369283+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1199, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2980, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1199}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20941, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.840898+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:02.840898+0000', 'last_peered': '2024-09-11T01:25:02.840898+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:02.840898+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839646+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:02.839646+0000', 'last_peered': '2024-09-11T01:25:02.839646+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:02.839646+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:22.983 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839253+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:25:02.839253+0000', 'last_peered': '2024-09-11T01:25:02.839253+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:02.839253+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:22.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:23.385 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:23.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29742/24 objects degraded (123925.000%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":723,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1503133696,"bytes_avail":288407158784,"bytes_total":289910292480,"degraded_objects":29742,"degraded_total":24,"degraded_ratio":1239.25,"recovering_objects_per_sec":177,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2133,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:23.399 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:23.400 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:23.757 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:23.758 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:23.771 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-11T01:25:22.663761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19893,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2133,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002430"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839185+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839185+0000","last_peered":"2024-09-11T01:25:02.839185+0000","last_clean":"2024-09-11T01:25:02.839185+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839185+0000","last_undegraded":"2024-09-11T01:25:02.839185+0000","last_fullsized":"2024-09-11T01:25:02.839185+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842776+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842776+0000","last_peered":"2024-09-11T01:25:02.842776+0000","last_clean":"2024-09-11T01:25:02.842776+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842776+0000","last_undegraded":"2024-09-11T01:25:02.842776+0000","last_fullsized":"2024-09-11T01:25:02.842776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":26627,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:21.369283+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:21.369283+0000","last_peered":"2024-09-11T01:25:21.369283+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:21.369283+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2980,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1199}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19892,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790073,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:26.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:27.093 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:27.094 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:27.106 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T01:25:26.921405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21018,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.258805"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839185+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839185+0000","last_peered":"2024-09-11T01:25:02.839185+0000","last_clean":"2024-09-11T01:25:02.839185+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839185+0000","last_undegraded":"2024-09-11T01:25:02.839185+0000","last_fullsized":"2024-09-11T01:25:02.839185+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842776+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842776+0000","last_peered":"2024-09-11T01:25:02.842776+0000","last_clean":"2024-09-11T01:25:02.842776+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842776+0000","last_undegraded":"2024-09-11T01:25:02.842776+0000","last_fullsized":"2024-09-11T01:25:02.842776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":28877,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:26.370165+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:26.370165+0000","last_peered":"2024-09-11T01:25:26.370165+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:26.370165+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-74}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21017,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790073,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:27.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:27.484 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:27.485 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:27.498 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T01:25:26.921405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21018,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467904,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1268083,"kb_avail":281647616,"statfs":{"total":289910292480,"available":288407158784,"internally_reserved":0,"allocated":204541952,"data_stored":203608470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1298516999},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.258805"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839185+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839185+0000","last_peered":"2024-09-11T01:25:02.839185+0000","last_clean":"2024-09-11T01:25:02.839185+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839185+0000","last_undegraded":"2024-09-11T01:25:02.839185+0000","last_fullsized":"2024-09-11T01:25:02.839185+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842776+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:02.842776+0000","last_peered":"2024-09-11T01:25:02.842776+0000","last_clean":"2024-09-11T01:25:02.842776+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:02.842776+0000","last_undegraded":"2024-09-11T01:25:02.842776+0000","last_fullsized":"2024-09-11T01:25:02.842776+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28602,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842503+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:02.842503+0000","last_peered":"2024-09-11T01:25:02.842503+0000","last_clean":"2024-09-11T01:25:02.842503+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:02.842503+0000","last_undegraded":"2024-09-11T01:25:02.842503+0000","last_fullsized":"2024-09-11T01:25:02.842503+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842589+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:02.842589+0000","last_peered":"2024-09-11T01:25:02.842589+0000","last_clean":"2024-09-11T01:25:02.842589+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:02.842589+0000","last_undegraded":"2024-09-11T01:25:02.842589+0000","last_fullsized":"2024-09-11T01:25:02.842589+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":28877,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:26.370165+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:26.370165+0000","last_peered":"2024-09-11T01:25:26.370165+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:26.370165+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-74}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28833,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842696+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:02.842696+0000","last_peered":"2024-09-11T01:25:02.842696+0000","last_clean":"2024-09-11T01:25:02.842696+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:02.842696+0000","last_undegraded":"2024-09-11T01:25:02.842696+0000","last_fullsized":"2024-09-11T01:25:02.842696+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842571+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:02.842571+0000","last_peered":"2024-09-11T01:25:02.842571+0000","last_clean":"2024-09-11T01:25:02.842571+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:02.842571+0000","last_undegraded":"2024-09-11T01:25:02.842571+0000","last_fullsized":"2024-09-11T01:25:02.842571+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842790+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:02.842790+0000","last_peered":"2024-09-11T01:25:02.842790+0000","last_clean":"2024-09-11T01:25:02.842790+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:02.842790+0000","last_undegraded":"2024-09-11T01:25:02.842790+0000","last_fullsized":"2024-09-11T01:25:02.842790+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.842483+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:02.842483+0000","last_peered":"2024-09-11T01:25:02.842483+0000","last_clean":"2024-09-11T01:25:02.842483+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:02.842483+0000","last_undegraded":"2024-09-11T01:25:02.842483+0000","last_fullsized":"2024-09-11T01:25:02.842483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21017,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203354112,"data_stored":203216818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790073,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483356,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888484,"statfs":{"total":96636764160,"available":96141807616,"internally_reserved":0,"allocated":471040,"data_stored":139251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":8694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:27.499 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:27.499 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.841309+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:02.841309+0000', 'last_peered': '2024-09-11T01:25:02.841309+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:02.841309+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20484, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.843770+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:02.843770+0000', 'last_peered': '2024-09-11T01:25:02.843770+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:02.843770+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21110, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839912+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:02.839912+0000', 'last_peered': '2024-09-11T01:25:02.839912+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:02.839912+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20883, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839073+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:02.839073+0000', 'last_peered': '2024-09-11T01:25:02.839073+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:02.839073+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:27.500 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 28877, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:26.370165+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:26.370165+0000', 'last_peered': '2024-09-11T01:25:26.370165+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:26.370165+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 74, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4105, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -74}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20941, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.840898+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:02.840898+0000', 'last_peered': '2024-09-11T01:25:02.840898+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:02.840898+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20910, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839646+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:02.839646+0000', 'last_peered': '2024-09-11T01:25:02.839646+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:02.839646+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:27.501 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 20387, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.839253+0000', 'last_change': '2024-09-11T01:15:20.504940+0000', 'last_active': '2024-09-11T01:25:02.839253+0000', 'last_peered': '2024-09-11T01:25:02.839253+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:02.839253+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4104, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 52, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4100}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:27.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:27.925 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:27.926 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28617/24 objects degraded (119237.500%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":728,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1503133696,"bytes_avail":288407158784,"bytes_total":289910292480,"degraded_objects":28617,"degraded_total":24,"degraded_ratio":1192.375,"recovering_objects_per_sec":221,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2653,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:27.945 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:27.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:28.355 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:28.355 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:28.369 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-11T01:25:27.926316+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21018,"num_bytes_recovered":36905945,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476228,"kb_used_data":199748,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639292,"statfs":{"total":289910292480,"available":288398635008,"internally_reserved":0,"allocated":204541952,"data_stored":203610406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1963,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1963,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.263377"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28575,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.908645+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:02.908645+0000","last_peered":"2024-09-11T01:25:02.908645+0000","last_clean":"2024-09-11T01:25:02.908645+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:02.908645+0000","last_undegraded":"2024-09-11T01:25:02.908645+0000","last_fullsized":"2024-09-11T01:25:02.908645+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20409,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.839185+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:02.839185+0000","last_peered":"2024-09-11T01:25:02.839185+0000","last_clean":"2024-09-11T01:25:02.839185+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:02.839185+0000","last_undegraded":"2024-09-11T01:25:02.839185+0000","last_fullsized":"2024-09-11T01:25:02.839185+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20484,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.843770+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:02.843770+0000","last_peered":"2024-09-11T01:25:02.843770+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:02.843770+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839912+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:02.839912+0000","last_peered":"2024-09-11T01:25:02.839912+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:02.839912+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838969+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:02.838969+0000","last_peered":"2024-09-11T01:25:02.838969+0000","last_clean":"2024-09-11T01:25:02.838969+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:02.838969+0000","last_undegraded":"2024-09-11T01:25:02.838969+0000","last_fullsized":"2024-09-11T01:25:02.838969+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923824+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:26.923824+0000","last_peered":"2024-09-11T01:25:26.923824+0000","last_clean":"2024-09-11T01:25:26.923824+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:26.923824+0000","last_undegraded":"2024-09-11T01:25:26.923824+0000","last_fullsized":"2024-09-11T01:25:26.923824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923849+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:26.923849+0000","last_peered":"2024-09-11T01:25:26.923849+0000","last_clean":"2024-09-11T01:25:26.923849+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:26.923849+0000","last_undegraded":"2024-09-11T01:25:26.923849+0000","last_fullsized":"2024-09-11T01:25:26.923849+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20883,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839073+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:02.839073+0000","last_peered":"2024-09-11T01:25:02.839073+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:02.839073+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":108,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.838955+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:02.838955+0000","last_peered":"2024-09-11T01:25:02.838955+0000","last_clean":"2024-09-11T01:25:02.838955+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:02.838955+0000","last_undegraded":"2024-09-11T01:25:02.838955+0000","last_fullsized":"2024-09-11T01:25:02.838955+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923759+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:26.923759+0000","last_peered":"2024-09-11T01:25:26.923759+0000","last_clean":"2024-09-11T01:25:26.923759+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:26.923759+0000","last_undegraded":"2024-09-11T01:25:26.923759+0000","last_fullsized":"2024-09-11T01:25:26.923759+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":28877,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:26.370165+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:26.370165+0000","last_peered":"2024-09-11T01:25:26.370165+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:26.370165+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-74}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28835,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923681+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:26.923681+0000","last_peered":"2024-09-11T01:25:26.923681+0000","last_clean":"2024-09-11T01:25:26.923681+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:26.923681+0000","last_undegraded":"2024-09-11T01:25:26.923681+0000","last_fullsized":"2024-09-11T01:25:26.923681+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923375+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:26.923375+0000","last_peered":"2024-09-11T01:25:26.923375+0000","last_clean":"2024-09-11T01:25:26.923375+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:26.923375+0000","last_undegraded":"2024-09-11T01:25:26.923375+0000","last_fullsized":"2024-09-11T01:25:26.923375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.840898+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:02.840898+0000","last_peered":"2024-09-11T01:25:02.840898+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:02.840898+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20910,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839646+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:02.839646+0000","last_peered":"2024-09-11T01:25:02.839646+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:02.839646+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923853+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:26.923853+0000","last_peered":"2024-09-11T01:25:26.923853+0000","last_clean":"2024-09-11T01:25:26.923853+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:26.923853+0000","last_undegraded":"2024-09-11T01:25:26.923853+0000","last_fullsized":"2024-09-11T01:25:26.923853+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923610+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:26.923610+0000","last_peered":"2024-09-11T01:25:26.923610+0000","last_clean":"2024-09-11T01:25:26.923610+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:26.923610+0000","last_undegraded":"2024-09-11T01:25:26.923610+0000","last_fullsized":"2024-09-11T01:25:26.923610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20387,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.839253+0000","last_change":"2024-09-11T01:15:20.504940+0000","last_active":"2024-09-11T01:25:02.839253+0000","last_peered":"2024-09-11T01:25:02.839253+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:02.839253+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":52,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4100},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21017,"num_bytes_recovered":36905926,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790074,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":471040,"data_stored":141187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508328,"kb_used_data":65748,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863512,"statfs":{"total":96636764160,"available":96116236288,"internally_reserved":0,"allocated":67325952,"data_stored":67023067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:31.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:31.701 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:31.701 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:31.713 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-11T01:25:30.665199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21372,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476248,"kb_used_data":199764,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639272,"statfs":{"total":289910292480,"available":288398614528,"internally_reserved":0,"allocated":204558336,"data_stored":203616189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001437"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923824+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:26.923824+0000","last_peered":"2024-09-11T01:25:26.923824+0000","last_clean":"2024-09-11T01:25:26.923824+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:26.923824+0000","last_undegraded":"2024-09-11T01:25:26.923824+0000","last_fullsized":"2024-09-11T01:25:26.923824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923849+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:26.923849+0000","last_peered":"2024-09-11T01:25:26.923849+0000","last_clean":"2024-09-11T01:25:26.923849+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:26.923849+0000","last_undegraded":"2024-09-11T01:25:26.923849+0000","last_fullsized":"2024-09-11T01:25:26.923849+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923759+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:26.923759+0000","last_peered":"2024-09-11T01:25:26.923759+0000","last_clean":"2024-09-11T01:25:26.923759+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:26.923759+0000","last_undegraded":"2024-09-11T01:25:26.923759+0000","last_fullsized":"2024-09-11T01:25:26.923759+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":28877,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:26.370165+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:26.370165+0000","last_peered":"2024-09-11T01:25:26.370165+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:26.370165+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-74}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28835,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923681+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:26.923681+0000","last_peered":"2024-09-11T01:25:26.923681+0000","last_clean":"2024-09-11T01:25:26.923681+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:26.923681+0000","last_undegraded":"2024-09-11T01:25:26.923681+0000","last_fullsized":"2024-09-11T01:25:26.923681+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923375+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:26.923375+0000","last_peered":"2024-09-11T01:25:26.923375+0000","last_clean":"2024-09-11T01:25:26.923375+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:26.923375+0000","last_undegraded":"2024-09-11T01:25:26.923375+0000","last_fullsized":"2024-09-11T01:25:26.923375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923853+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:26.923853+0000","last_peered":"2024-09-11T01:25:26.923853+0000","last_clean":"2024-09-11T01:25:26.923853+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:26.923853+0000","last_undegraded":"2024-09-11T01:25:26.923853+0000","last_fullsized":"2024-09-11T01:25:26.923853+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923610+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:26.923610+0000","last_peered":"2024-09-11T01:25:26.923610+0000","last_clean":"2024-09-11T01:25:26.923610+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:26.923610+0000","last_undegraded":"2024-09-11T01:25:26.923610+0000","last_fullsized":"2024-09-11T01:25:26.923610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":21100,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:29.058371+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:29.058371+0000","last_peered":"2024-09-11T01:25:29.058371+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:29.058371+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":406,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21371,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790074,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":471040,"data_stored":141187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:31.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:32.025 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:32.025 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:32.038 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-09-11T01:25:30.665199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21372,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476248,"kb_used_data":199764,"kb_used_omap":12,"kb_used_meta":1276403,"kb_avail":281639272,"statfs":{"total":289910292480,"available":288398614528,"internally_reserved":0,"allocated":204558336,"data_stored":203616189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1307036679},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1479,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001437"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20612,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841605+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:02.841605+0000","last_peered":"2024-09-11T01:25:02.841605+0000","last_clean":"2024-09-11T01:25:02.841605+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:02.841605+0000","last_undegraded":"2024-09-11T01:25:02.841605+0000","last_fullsized":"2024-09-11T01:25:02.841605+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20224,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:02.841309+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:02.841309+0000","last_peered":"2024-09-11T01:25:02.841309+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:02.841309+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841391+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:02.841391+0000","last_peered":"2024-09-11T01:25:02.841391+0000","last_clean":"2024-09-11T01:25:02.841391+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:02.841391+0000","last_undegraded":"2024-09-11T01:25:02.841391+0000","last_fullsized":"2024-09-11T01:25:02.841391+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923824+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:26.923824+0000","last_peered":"2024-09-11T01:25:26.923824+0000","last_clean":"2024-09-11T01:25:26.923824+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:26.923824+0000","last_undegraded":"2024-09-11T01:25:26.923824+0000","last_fullsized":"2024-09-11T01:25:26.923824+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28604,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923849+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:26.923849+0000","last_peered":"2024-09-11T01:25:26.923849+0000","last_clean":"2024-09-11T01:25:26.923849+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:26.923849+0000","last_undegraded":"2024-09-11T01:25:26.923849+0000","last_fullsized":"2024-09-11T01:25:26.923849+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841061+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:02.841061+0000","last_peered":"2024-09-11T01:25:02.841061+0000","last_clean":"2024-09-11T01:25:02.841061+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:02.841061+0000","last_undegraded":"2024-09-11T01:25:02.841061+0000","last_fullsized":"2024-09-11T01:25:02.841061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28307,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841143+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:02.841143+0000","last_peered":"2024-09-11T01:25:02.841143+0000","last_clean":"2024-09-11T01:25:02.841143+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:02.841143+0000","last_undegraded":"2024-09-11T01:25:02.841143+0000","last_fullsized":"2024-09-11T01:25:02.841143+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923759+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:26.923759+0000","last_peered":"2024-09-11T01:25:26.923759+0000","last_clean":"2024-09-11T01:25:26.923759+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:26.923759+0000","last_undegraded":"2024-09-11T01:25:26.923759+0000","last_fullsized":"2024-09-11T01:25:26.923759+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":28877,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:26.370165+0000","last_change":"2024-09-11T01:25:00.818520+0000","last_active":"2024-09-11T01:25:26.370165+0000","last_peered":"2024-09-11T01:25:26.370165+0000","last_clean":"2024-09-11T01:15:17.954818+0000","last_became_active":"2024-09-11T01:15:20.515940+0000","last_became_peered":"2024-09-11T01:15:20.515940+0000","last_unstale":"2024-09-11T01:25:26.370165+0000","last_undegraded":"2024-09-11T01:15:20.037374+0000","last_fullsized":"2024-09-11T01:15:19.987002+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":74,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4105,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-74}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28835,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923681+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:26.923681+0000","last_peered":"2024-09-11T01:25:26.923681+0000","last_clean":"2024-09-11T01:25:26.923681+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:26.923681+0000","last_undegraded":"2024-09-11T01:25:26.923681+0000","last_fullsized":"2024-09-11T01:25:26.923681+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923375+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:26.923375+0000","last_peered":"2024-09-11T01:25:26.923375+0000","last_clean":"2024-09-11T01:25:26.923375+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:26.923375+0000","last_undegraded":"2024-09-11T01:25:26.923375+0000","last_fullsized":"2024-09-11T01:25:26.923375+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:25:02.841499+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:02.841499+0000","last_peered":"2024-09-11T01:25:02.841499+0000","last_clean":"2024-09-11T01:25:02.841499+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:02.841499+0000","last_undegraded":"2024-09-11T01:25:02.841499+0000","last_fullsized":"2024-09-11T01:25:02.841499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923853+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:26.923853+0000","last_peered":"2024-09-11T01:25:26.923853+0000","last_clean":"2024-09-11T01:25:26.923853+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:26.923853+0000","last_undegraded":"2024-09-11T01:25:26.923853+0000","last_fullsized":"2024-09-11T01:25:26.923853+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:25:26.923610+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:26.923610+0000","last_peered":"2024-09-11T01:25:26.923610+0000","last_clean":"2024-09-11T01:25:26.923610+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:26.923610+0000","last_undegraded":"2024-09-11T01:25:26.923610+0000","last_fullsized":"2024-09-11T01:25:26.923610+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":21100,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:29.058371+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:29.058371+0000","last_peered":"2024-09-11T01:25:29.058371+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:29.058371+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":406,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21371,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476220,"kb_used_data":133540,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895620,"statfs":{"total":96636764160,"available":96149114880,"internally_reserved":0,"allocated":136744960,"data_stored":136446152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790074,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":471040,"data_stored":141187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:32.039 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:32.039 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20224, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:02.841309+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:02.841309+0000', 'last_peered': '2024-09-11T01:25:02.841309+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:02.841309+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:32.039 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20887, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930524+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.930524+0000', 'last_peered': '2024-09-11T01:25:27.930524+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.930524+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "45'20562", 'reported_seq': 28877, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:26.370165+0000', 'last_change': '2024-09-11T01:25:00.818520+0000', 'last_active': '2024-09-11T01:25:26.370165+0000', 'last_peered': '2024-09-11T01:25:26.370165+0000', 'last_clean': '2024-09-11T01:15:17.954818+0000', 'last_became_active': '2024-09-11T01:15:20.515940+0000', 'last_became_peered': '2024-09-11T01:15:20.515940+0000', 'last_unstale': '2024-09-11T01:25:26.370165+0000', 'last_undegraded': '2024-09-11T01:15:20.037374+0000', 'last_fullsized': '2024-09-11T01:15:19.987002+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'2404", 'last_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:54.063207+0000', 'objects_scrubbed': 2357, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 785, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 74, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18199, 'num_write_kb': 31684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4105, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -74}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:32.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:32.041 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:32.041 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:32.041 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 21100, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:29.058371+0000', 'last_change': '2024-09-11T01:25:26.627152+0000', 'last_active': '2024-09-11T01:25:29.058371+0000', 'last_peered': '2024-09-11T01:25:29.058371+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:29.058371+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3750, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 406, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:32.041 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:32.434 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:32.435 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28263/24 objects degraded (117762.500%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":732,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1511677952,"bytes_avail":288398614528,"bytes_total":289910292480,"degraded_objects":28263,"degraded_total":24,"degraded_ratio":1177.625,"recovering_objects_per_sec":184,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1479,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:32.448 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:32.448 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:32.807 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:32.807 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:32.819 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-11T01:25:32.665502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21446,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1553,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1553,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001390"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":21100,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:29.058371+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:29.058371+0000","last_peered":"2024-09-11T01:25:29.058371+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:29.058371+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":406,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21445,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790075,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:35.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:36.171 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:36.172 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:36.184 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-11T01:25:34.665990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22000,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":982,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001520"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":22209,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:34.060922+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:34.060922+0000","last_peered":"2024-09-11T01:25:34.060922+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:34.060922+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21999,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790075,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:36.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:36.515 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:36.516 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:36.528 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-11T01:25:34.665990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22000,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":982,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001520"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":22209,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:34.060922+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:34.060922+0000","last_peered":"2024-09-11T01:25:34.060922+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:34.060922+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21999,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790075,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:36.529 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20228, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.929186+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:27.929186+0000', 'last_peered': '2024-09-11T01:25:27.929186+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:27.929186+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20887, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930524+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.930524+0000', 'last_peered': '2024-09-11T01:25:27.930524+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.930524+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:36.530 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:36.531 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:36.531 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 22209, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:34.060922+0000', 'last_change': '2024-09-11T01:25:26.627152+0000', 'last_active': '2024-09-11T01:25:34.060922+0000', 'last_peered': '2024-09-11T01:25:34.060922+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:34.060922+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3196, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 960, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:36.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:36.947 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:36.947 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27635/24 objects degraded (115145.833%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":737,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1517400064,"bytes_avail":288392892416,"bytes_total":289910292480,"degraded_objects":27635,"degraded_total":24,"degraded_ratio":1151.4583333333333,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":982,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:36.960 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:36.961 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:37.291 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:37.291 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:37.304 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-11T01:25:36.666333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22000,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":982,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.744927"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":22209,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:34.060922+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:34.060922+0000","last_peered":"2024-09-11T01:25:34.060922+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:34.060922+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21999,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790075,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:40.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:40.616 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:40.617 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:40.630 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-11T01:25:38.666675+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22000,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485996,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629524,"statfs":{"total":289910292480,"available":288388632576,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":982,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.740359"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20887,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930524+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.930524+0000","last_peered":"2024-09-11T01:25:27.930524+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.930524+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":22209,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:34.060922+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:34.060922+0000","last_peered":"2024-09-11T01:25:34.060922+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:34.060922+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":960,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27635,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21999,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790076,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:40.631 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:40.936 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:40.937 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:40.948 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-11T01:25:40.667007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22602,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485996,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629524,"statfs":{"total":289910292480,"available":288388632576,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1584,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002152"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":23413,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:39.065676+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:39.065676+0000","last_peered":"2024-09-11T01:25:39.065676+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:39.065676+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22601,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790076,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:40.951 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:40.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20228, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.929186+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:27.929186+0000', 'last_peered': '2024-09-11T01:25:27.929186+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:27.929186+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:40.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:40.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:40.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:40.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:40.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:40.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20888, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931530+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.931530+0000', 'last_peered': '2024-09-11T01:25:27.931530+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.931530+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:40.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:40.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:40.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:40.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:40.954 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:40.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 23413, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:39.065676+0000', 'last_change': '2024-09-11T01:25:26.627152+0000', 'last_active': '2024-09-11T01:25:39.065676+0000', 'last_peered': '2024-09-11T01:25:39.065676+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:39.065676+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2594, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1562, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:40.955 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:41.352 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:41.352 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27033/24 objects degraded (112637.500%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":741,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1521659904,"bytes_avail":288388632576,"bytes_total":289910292480,"degraded_objects":27033,"degraded_total":24,"degraded_ratio":1126.375,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":40,"recovering_keys_per_sec":0,"num_objects_recovered":1584,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:41.366 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:41.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:41.757 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:41.757 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:41.770 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-09-11T01:25:40.667007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22602,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485996,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629524,"statfs":{"total":289910292480,"available":288388632576,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1584,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002152"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20616,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930231+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930231+0000","last_peered":"2024-09-11T01:25:27.930231+0000","last_clean":"2024-09-11T01:25:27.930231+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930231+0000","last_undegraded":"2024-09-11T01:25:27.930231+0000","last_fullsized":"2024-09-11T01:25:27.930231+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929028+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929028+0000","last_peered":"2024-09-11T01:25:27.929028+0000","last_clean":"2024-09-11T01:25:27.929028+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929028+0000","last_undegraded":"2024-09-11T01:25:27.929028+0000","last_fullsized":"2024-09-11T01:25:27.929028+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":23413,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:39.065676+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:39.065676+0000","last_peered":"2024-09-11T01:25:39.065676+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:39.065676+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1562,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2592}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27033,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22601,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790076,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:44.773 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:45.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:45.100 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:45.112 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-11T01:25:44.667774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23288,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490156,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625364,"statfs":{"total":289910292480,"available":288384372736,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002272"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":24785,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:44.067115+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:44.067115+0000","last_peered":"2024-09-11T01:25:44.067115+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:44.067115+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23287,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790077,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:45.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:45.430 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:45.430 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:45.442 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-11T01:25:44.667774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23288,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490156,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625364,"statfs":{"total":289910292480,"available":288384372736,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002272"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":24785,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:44.067115+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:44.067115+0000","last_peered":"2024-09-11T01:25:44.067115+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:44.067115+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23287,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790077,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20228, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.929186+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:27.929186+0000', 'last_peered': '2024-09-11T01:25:27.929186+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:27.929186+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20888, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931530+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.931530+0000', 'last_peered': '2024-09-11T01:25:27.931530+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.931530+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:45.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:45.445 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:45.445 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:45.445 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:45.445 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 24785, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:44.067115+0000', 'last_change': '2024-09-11T01:25:26.627152+0000', 'last_active': '2024-09-11T01:25:44.067115+0000', 'last_peered': '2024-09-11T01:25:44.067115+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:44.067115+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1908, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2248, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:45.445 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:45.824 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:45.824 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26347/24 objects degraded (109779.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":746,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1525919744,"bytes_avail":288384372736,"bytes_total":289910292480,"degraded_objects":26347,"degraded_total":24,"degraded_ratio":1097.7916666666667,"recovering_objects_per_sec":153,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1842,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:45.837 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:45.837 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:46.145 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:46.146 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:46.158 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-11T01:25:44.667774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23288,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490156,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625364,"statfs":{"total":289910292480,"available":288384372736,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1842,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002272"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929113+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929113+0000","last_peered":"2024-09-11T01:25:27.929113+0000","last_clean":"2024-09-11T01:25:27.929113+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929113+0000","last_undegraded":"2024-09-11T01:25:27.929113+0000","last_fullsized":"2024-09-11T01:25:27.929113+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929789+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929789+0000","last_peered":"2024-09-11T01:25:27.929789+0000","last_clean":"2024-09-11T01:25:27.929789+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929789+0000","last_undegraded":"2024-09-11T01:25:27.929789+0000","last_fullsized":"2024-09-11T01:25:27.929789+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":24785,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:44.067115+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:44.067115+0000","last_peered":"2024-09-11T01:25:44.067115+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:44.067115+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23287,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790077,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:49.161 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:49.496 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:49.497 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:49.513 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-11T01:25:48.668497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23288,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002164"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":24785,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:44.067115+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:44.067115+0000","last_peered":"2024-09-11T01:25:44.067115+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:44.067115+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23287,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790078,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:49.514 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:49.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:49.837 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:49.850 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-11T01:25:48.668497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23288,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002164"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":24785,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:44.067115+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:44.067115+0000","last_peered":"2024-09-11T01:25:44.067115+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:44.067115+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23287,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790078,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:49.851 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:49.851 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20228, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.929186+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:27.929186+0000', 'last_peered': '2024-09-11T01:25:27.929186+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:27.929186+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:49.851 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:49.851 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20888, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931530+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.931530+0000', 'last_peered': '2024-09-11T01:25:27.931530+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.931530+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:49.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 24785, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:44.067115+0000', 'last_change': '2024-09-11T01:25:26.627152+0000', 'last_active': '2024-09-11T01:25:44.067115+0000', 'last_peered': '2024-09-11T01:25:44.067115+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:44.067115+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1908, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2248, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:49.853 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:50.237 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:50.238 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26347/24 objects degraded (109779.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":750,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1517400064,"bytes_avail":288392892416,"bytes_total":289910292480,"degraded_objects":26347,"degraded_total":24,"degraded_ratio":1097.7916666666667,"recovering_objects_per_sec":107,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1288,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:50.251 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:50.251 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:50.557 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:50.557 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:50.571 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-11T01:25:48.668497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23288,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481836,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633684,"statfs":{"total":289910292480,"available":288392892416,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1288,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002164"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928833+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928833+0000","last_peered":"2024-09-11T01:25:27.928833+0000","last_clean":"2024-09-11T01:25:27.928833+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928833+0000","last_undegraded":"2024-09-11T01:25:27.928833+0000","last_fullsized":"2024-09-11T01:25:27.928833+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":24785,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:44.067115+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:44.067115+0000","last_peered":"2024-09-11T01:25:44.067115+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:44.067115+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2248,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23287,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790078,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497252,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874588,"statfs":{"total":96636764160,"available":96127578112,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:53.574 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:53.877 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:53.877 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:53.891 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-11T01:25:52.669191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24182,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490156,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625364,"statfs":{"total":289910292480,"available":288384372736,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1580,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002184"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928877+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928877+0000","last_peered":"2024-09-11T01:25:27.928877+0000","last_clean":"2024-09-11T01:25:27.928877+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928877+0000","last_undegraded":"2024-09-11T01:25:27.928877+0000","last_fullsized":"2024-09-11T01:25:27.928877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":26573,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:49.066128+0000","last_change":"2024-09-11T01:25:26.627152+0000","last_active":"2024-09-11T01:25:49.066128+0000","last_peered":"2024-09-11T01:25:49.066128+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:49.066128+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3142,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1012}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25453,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24181,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:53.892 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:54.215 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:54.215 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:54.228 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-11T01:25:54.088106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24439,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25196,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490156,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625364,"statfs":{"total":289910292480,"available":288384372736,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2594,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2594,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.420712"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928877+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928877+0000","last_peered":"2024-09-11T01:25:27.928877+0000","last_clean":"2024-09-11T01:25:27.928877+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928877+0000","last_undegraded":"2024-09-11T01:25:27.928877+0000","last_fullsized":"2024-09-11T01:25:27.928877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":28602,"reported_epoch":49,"state":"active+undersized+remapped","last_fresh":"2024-09-11T01:25:53.478151+0000","last_change":"2024-09-11T01:25:53.478360+0000","last_active":"2024-09-11T01:25:53.478151+0000","last_peered":"2024-09-11T01:25:53.478151+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:53.478151+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24439,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25195,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 20228, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.929186+0000', 'last_change': '2024-09-11T01:15:20.525508+0000', 'last_active': '2024-09-11T01:25:27.929186+0000', 'last_peered': '2024-09-11T01:25:27.929186+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:27.929186+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3964, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 43, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3963}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20888, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931530+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.931530+0000', 'last_peered': '2024-09-11T01:25:27.931530+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.931530+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:54.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:54.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:54.231 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:54.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:54.231 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:54.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'20138", 'reported_seq': 28602, 'reported_epoch': 49, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-11T01:25:53.478151+0000', 'last_change': '2024-09-11T01:25:53.478360+0000', 'last_active': '2024-09-11T01:25:53.478151+0000', 'last_peered': '2024-09-11T01:25:53.478151+0000', 'last_clean': '2024-09-11T01:15:17.933954+0000', 'last_became_active': '2024-09-11T01:15:20.498848+0000', 'last_became_peered': '2024-09-11T01:15:20.498848+0000', 'last_unstale': '2024-09-11T01:25:53.478151+0000', 'last_undegraded': '2024-09-11T01:15:20.079809+0000', 'last_fullsized': '2024-09-11T01:15:19.985513+0000', 'mapping_epoch': 38, 'log_start': "41'10100", 'ondisk_log_start': "41'10100", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3390", 'last_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:05.985745+0000', 'objects_scrubbed': 3333, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1094, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16795, 'num_write_kb': 26928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4156, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '1', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:54.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:54.681 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:54.682 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25453/24 objects degraded (106054.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":755,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1525919744,"bytes_avail":288384372736,"bytes_total":289910292480,"degraded_objects":25453,"degraded_total":24,"degraded_ratio":1060.5416666666667,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1580,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:54.696 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:54.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:55.033 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:55.034 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:55.046 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-11T01:25:54.669638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24439,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25196,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490156,"kb_used_data":199788,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625364,"statfs":{"total":289910292480,"available":288384372736,"internally_reserved":0,"allocated":204582912,"data_stored":203624853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1908,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1908,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001864"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930275+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:27.930275+0000","last_peered":"2024-09-11T01:25:27.930275+0000","last_clean":"2024-09-11T01:25:27.930275+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:27.930275+0000","last_undegraded":"2024-09-11T01:25:27.930275+0000","last_fullsized":"2024-09-11T01:25:27.930275+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20228,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.929186+0000","last_change":"2024-09-11T01:15:20.525508+0000","last_active":"2024-09-11T01:25:27.929186+0000","last_peered":"2024-09-11T01:25:27.929186+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:27.929186+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3963}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929069+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:27.929069+0000","last_peered":"2024-09-11T01:25:27.929069+0000","last_clean":"2024-09-11T01:25:27.929069+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:27.929069+0000","last_undegraded":"2024-09-11T01:25:27.929069+0000","last_fullsized":"2024-09-11T01:25:27.929069+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928462+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:27.928462+0000","last_peered":"2024-09-11T01:25:27.928462+0000","last_clean":"2024-09-11T01:25:27.928462+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:27.928462+0000","last_undegraded":"2024-09-11T01:25:27.928462+0000","last_fullsized":"2024-09-11T01:25:27.928462+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28311,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:27.930075+0000","last_peered":"2024-09-11T01:25:27.930075+0000","last_clean":"2024-09-11T01:25:27.930075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:27.930075+0000","last_undegraded":"2024-09-11T01:25:27.930075+0000","last_fullsized":"2024-09-11T01:25:27.930075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29036,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:28.008281+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:28.008281+0000","last_peered":"2024-09-11T01:25:28.008281+0000","last_clean":"2024-09-11T01:25:28.008281+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:28.008281+0000","last_undegraded":"2024-09-11T01:25:28.008281+0000","last_fullsized":"2024-09-11T01:25:28.008281+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930094+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:27.930094+0000","last_peered":"2024-09-11T01:25:27.930094+0000","last_clean":"2024-09-11T01:25:27.930094+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:27.930094+0000","last_undegraded":"2024-09-11T01:25:27.930094+0000","last_fullsized":"2024-09-11T01:25:27.930094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928877+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928877+0000","last_peered":"2024-09-11T01:25:27.928877+0000","last_clean":"2024-09-11T01:25:27.928877+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928877+0000","last_undegraded":"2024-09-11T01:25:27.928877+0000","last_fullsized":"2024-09-11T01:25:27.928877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":28602,"reported_epoch":49,"state":"active+undersized+remapped","last_fresh":"2024-09-11T01:25:53.478151+0000","last_change":"2024-09-11T01:25:53.478360+0000","last_active":"2024-09-11T01:25:53.478151+0000","last_peered":"2024-09-11T01:25:53.478151+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:53.478151+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24439,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25195,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476236,"kb_used_data":133556,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895604,"statfs":{"total":96636764160,"available":96149098496,"internally_reserved":0,"allocated":136761344,"data_stored":136451935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:58.049 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:58.359 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:58.359 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:58.371 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-11T01:25:56.669956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25656,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490172,"kb_used_data":199804,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625348,"statfs":{"total":289910292480,"available":288384356352,"internally_reserved":0,"allocated":204599296,"data_stored":203630586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2368,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2368,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001459"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":21153,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:56.367924+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:25:56.367924+0000","last_peered":"2024-09-11T01:25:56.367924+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:56.367924+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3504,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929468+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:27.929468+0000","last_peered":"2024-09-11T01:25:27.929468+0000","last_clean":"2024-09-11T01:25:27.929468+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:27.929468+0000","last_undegraded":"2024-09-11T01:25:27.929468+0000","last_fullsized":"2024-09-11T01:25:27.929468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28606,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929209+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:27.929209+0000","last_peered":"2024-09-11T01:25:27.929209+0000","last_clean":"2024-09-11T01:25:27.929209+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:27.929209+0000","last_undegraded":"2024-09-11T01:25:27.929209+0000","last_fullsized":"2024-09-11T01:25:27.929209+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929159+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:27.929159+0000","last_peered":"2024-09-11T01:25:27.929159+0000","last_clean":"2024-09-11T01:25:27.929159+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:27.929159+0000","last_undegraded":"2024-09-11T01:25:27.929159+0000","last_fullsized":"2024-09-11T01:25:27.929159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28837,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929350+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:27.929350+0000","last_peered":"2024-09-11T01:25:27.929350+0000","last_clean":"2024-09-11T01:25:27.929350+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:27.929350+0000","last_undegraded":"2024-09-11T01:25:27.929350+0000","last_fullsized":"2024-09-11T01:25:27.929350+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929831+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:27.929831+0000","last_peered":"2024-09-11T01:25:27.929831+0000","last_clean":"2024-09-11T01:25:27.929831+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:27.929831+0000","last_undegraded":"2024-09-11T01:25:27.929831+0000","last_fullsized":"2024-09-11T01:25:27.929831+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.928877+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:27.928877+0000","last_peered":"2024-09-11T01:25:27.928877+0000","last_clean":"2024-09-11T01:25:27.928877+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:27.928877+0000","last_undegraded":"2024-09-11T01:25:27.928877+0000","last_fullsized":"2024-09-11T01:25:27.928877+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.929256+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:27.929256+0000","last_peered":"2024-09-11T01:25:27.929256+0000","last_clean":"2024-09-11T01:25:27.929256+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:27.929256+0000","last_undegraded":"2024-09-11T01:25:27.929256+0000","last_fullsized":"2024-09-11T01:25:27.929256+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":28602,"reported_epoch":49,"state":"active+undersized+remapped","last_fresh":"2024-09-11T01:25:53.478151+0000","last_change":"2024-09-11T01:25:53.478360+0000","last_active":"2024-09-11T01:25:53.478151+0000","last_peered":"2024-09-11T01:25:53.478151+0000","last_clean":"2024-09-11T01:15:17.933954+0000","last_became_active":"2024-09-11T01:15:20.498848+0000","last_became_peered":"2024-09-11T01:15:20.498848+0000","last_unstale":"2024-09-11T01:25:53.478151+0000","last_undegraded":"2024-09-11T01:15:20.079809+0000","last_fullsized":"2024-09-11T01:15:19.985513+0000","mapping_epoch":38,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25655,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203345920,"data_stored":203215852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505572,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866268,"statfs":{"total":96636764160,"available":96119058432,"internally_reserved":0,"allocated":479232,"data_stored":144068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:58.373 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:58.694 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:58.694 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:58.707 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-11T01:25:58.670284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25656,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481868,"kb_used_data":199816,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633652,"statfs":{"total":289910292480,"available":288392859648,"internally_reserved":0,"allocated":204611584,"data_stored":203635836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1474,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001421"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":21153,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:56.367924+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:25:56.367924+0000","last_peered":"2024-09-11T01:25:56.367924+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:56.367924+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3504,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28610,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122042+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122042+0000","last_peered":"2024-09-11T01:25:55.122042+0000","last_clean":"2024-09-11T01:25:55.122042+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122042+0000","last_undegraded":"2024-09-11T01:25:55.122042+0000","last_fullsized":"2024-09-11T01:25:55.122042+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25655,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790080,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:58.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:25:58.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 21153, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:56.367924+0000', 'last_change': '2024-09-11T01:25:53.478666+0000', 'last_active': '2024-09-11T01:25:56.367924+0000', 'last_peered': '2024-09-11T01:25:56.367924+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:25:56.367924+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3504, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 503, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3504}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:25:58.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:58.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20488, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930437+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:27.930437+0000', 'last_peered': '2024-09-11T01:25:27.930437+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:27.930437+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:58.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21114, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.930318+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:27.930318+0000', 'last_peered': '2024-09-11T01:25:27.930318+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:27.930318+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20888, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931530+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:27.931530+0000', 'last_peered': '2024-09-11T01:25:27.931530+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:27.931530+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20945, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931558+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:27.931558+0000', 'last_peered': '2024-09-11T01:25:27.931558+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:27.931558+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:25:58.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20914, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:27.931289+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:27.931289+0000', 'last_peered': '2024-09-11T01:25:27.931289+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:27.931289+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:58.710 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:59.092 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:59.092 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23979/24 objects degraded (99912.500%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":759,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1517432832,"bytes_avail":288392859648,"bytes_total":289910292480,"degraded_objects":23979,"degraded_total":24,"degraded_ratio":999.125,"recovering_objects_per_sec":184,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1474,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:59.105 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:59.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:59.472 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:25:59.473 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:25:59.485 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-11T01:25:58.670284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25656,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481868,"kb_used_data":199816,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633652,"statfs":{"total":289910292480,"available":288392859648,"internally_reserved":0,"allocated":204611584,"data_stored":203635836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1474,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001421"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28580,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.931323+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:27.931323+0000","last_peered":"2024-09-11T01:25:27.931323+0000","last_clean":"2024-09-11T01:25:27.931323+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:27.931323+0000","last_undegraded":"2024-09-11T01:25:27.931323+0000","last_fullsized":"2024-09-11T01:25:27.931323+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":21153,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:56.367924+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:25:56.367924+0000","last_peered":"2024-09-11T01:25:56.367924+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:25:56.367924+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3504,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3504}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20413,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930565+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:27.930565+0000","last_peered":"2024-09-11T01:25:27.930565+0000","last_clean":"2024-09-11T01:25:27.930565+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:27.930565+0000","last_undegraded":"2024-09-11T01:25:27.930565+0000","last_fullsized":"2024-09-11T01:25:27.930565+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20488,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930437+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:27.930437+0000","last_peered":"2024-09-11T01:25:27.930437+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:27.930437+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21114,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.930318+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:27.930318+0000","last_peered":"2024-09-11T01:25:27.930318+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:27.930318+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930153+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:27.930153+0000","last_peered":"2024-09-11T01:25:27.930153+0000","last_clean":"2024-09-11T01:25:27.930153+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:27.930153+0000","last_undegraded":"2024-09-11T01:25:27.930153+0000","last_fullsized":"2024-09-11T01:25:27.930153+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28610,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122042+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122042+0000","last_peered":"2024-09-11T01:25:55.122042+0000","last_clean":"2024-09-11T01:25:55.122042+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122042+0000","last_undegraded":"2024-09-11T01:25:55.122042+0000","last_fullsized":"2024-09-11T01:25:55.122042+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20888,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931530+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:27.931530+0000","last_peered":"2024-09-11T01:25:27.931530+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:27.931530+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:25:27.930301+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:27.930301+0000","last_peered":"2024-09-11T01:25:27.930301+0000","last_clean":"2024-09-11T01:25:27.930301+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:27.930301+0000","last_undegraded":"2024-09-11T01:25:27.930301+0000","last_fullsized":"2024-09-11T01:25:27.930301+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20945,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931558+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:27.931558+0000","last_peered":"2024-09-11T01:25:27.931558+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:27.931558+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20914,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:27.931289+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:27.931289+0000","last_peered":"2024-09-11T01:25:27.931289+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:27.931289+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25655,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790080,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497268,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874572,"statfs":{"total":96636764160,"available":96127561728,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508348,"kb_used_data":65764,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863492,"statfs":{"total":96636764160,"available":96116215808,"internally_reserved":0,"allocated":67342336,"data_stored":67028850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:02.487 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:02.810 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:02.810 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:02.822 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-09-11T01:26:02.670953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26237,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486040,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629480,"statfs":{"total":289910292480,"available":288388587520,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.582846"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":22315,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:01.371832+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:01.371832+0000","last_peered":"2024-09-11T01:26:01.371832+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:01.371832+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1084,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2923}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26236,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501424,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870416,"statfs":{"total":96636764160,"available":96123305984,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:02.824 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:03.177 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:03.178 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:03.191 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-09-11T01:26:02.670953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26237,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486040,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629480,"statfs":{"total":289910292480,"available":288388587520,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.582846"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":22315,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:01.371832+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:01.371832+0000","last_peered":"2024-09-11T01:26:01.371832+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:01.371832+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1084,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2923}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26236,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501424,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870416,"statfs":{"total":96636764160,"available":96123305984,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:03.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:26:03.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 22315, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:01.371832+0000', 'last_change': '2024-09-11T01:25:53.478666+0000', 'last_active': '2024-09-11T01:26:01.371832+0000', 'last_peered': '2024-09-11T01:26:01.371832+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:26:01.371832+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2923, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1084, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2923}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:26:03.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:03.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20492, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118032+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:55.118032+0000', 'last_peered': '2024-09-11T01:25:55.118032+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:55.118032+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:03.192 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:03.192 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118422+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:55.118422+0000', 'last_peered': '2024-09-11T01:25:55.118422+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:55.118422+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:03.193 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:03.193 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20892, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118086+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:55.118086+0000', 'last_peered': '2024-09-11T01:25:55.118086+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:55.118086+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:03.193 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:03.193 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20949, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118255+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:55.118255+0000', 'last_peered': '2024-09-11T01:25:55.118255+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:55.118255+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:03.193 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:03.193 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20918, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118244+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:55.118244+0000', 'last_peered': '2024-09-11T01:25:55.118244+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:55.118244+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:03.193 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:03.599 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:03.600 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23398/24 objects degraded (97491.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":764,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1521704960,"bytes_avail":288388587520,"bytes_total":289910292480,"degraded_objects":23398,"degraded_total":24,"degraded_ratio":974.91666666666652,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1041,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:03.613 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:03.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:03.930 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:03.930 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:03.943 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-09-11T01:26:02.670953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26237,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486040,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629480,"statfs":{"total":289910292480,"available":288388587520,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.582846"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":22315,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:01.371832+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:01.371832+0000","last_peered":"2024-09-11T01:26:01.371832+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:01.371832+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1084,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2923}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26236,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501424,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870416,"statfs":{"total":96636764160,"available":96123305984,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:06.945 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:07.276 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:07.277 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:07.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-11T01:26:06.671718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26881,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486040,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629480,"statfs":{"total":289910292480,"available":288388587520,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.554477"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":23604,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:06.371754+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:06.371754+0000","last_peered":"2024-09-11T01:26:06.371754+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:06.371754+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26880,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501424,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870416,"statfs":{"total":96636764160,"available":96123305984,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:07.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:07.605 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:07.605 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:07.616 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-11T01:26:06.671718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26881,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486040,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629480,"statfs":{"total":289910292480,"available":288388587520,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.554477"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":23604,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:06.371754+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:06.371754+0000","last_peered":"2024-09-11T01:26:06.371754+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:06.371754+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26880,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501424,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870416,"statfs":{"total":96636764160,"available":96123305984,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:07.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:26:07.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 23604, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:06.371754+0000', 'last_change': '2024-09-11T01:25:53.478666+0000', 'last_active': '2024-09-11T01:26:06.371754+0000', 'last_peered': '2024-09-11T01:26:06.371754+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:26:06.371754+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2279, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1728, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:26:07.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:07.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20492, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118032+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:55.118032+0000', 'last_peered': '2024-09-11T01:25:55.118032+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:55.118032+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118422+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:55.118422+0000', 'last_peered': '2024-09-11T01:25:55.118422+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:55.118422+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20892, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118086+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:55.118086+0000', 'last_peered': '2024-09-11T01:25:55.118086+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:55.118086+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20949, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118255+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:55.118255+0000', 'last_peered': '2024-09-11T01:25:55.118255+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:55.118255+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:07.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20918, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118244+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:55.118244+0000', 'last_peered': '2024-09-11T01:25:55.118244+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:55.118244+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:07.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:08.030 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:08.030 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22754/24 objects degraded (94808.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":768,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1521704960,"bytes_avail":288388587520,"bytes_total":289910292480,"degraded_objects":22754,"degraded_total":24,"degraded_ratio":948.08333333333348,"recovering_objects_per_sec":145,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1685,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:08.043 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:08.043 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:08.351 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:08.351 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:08.364 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-11T01:26:06.671718+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26881,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486040,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281629480,"statfs":{"total":289910292480,"available":288388587520,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1685,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1685,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.554477"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":23604,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:06.371754+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:06.371754+0000","last_peered":"2024-09-11T01:26:06.371754+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:06.371754+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26880,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790081,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501424,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870416,"statfs":{"total":96636764160,"available":96123305984,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:11.366 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:11.693 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:11.694 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:11.706 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-11T01:26:10.672375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26881,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490200,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625320,"statfs":{"total":289910292480,"available":288384327680,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002091"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":23604,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:06.371754+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:06.371754+0000","last_peered":"2024-09-11T01:26:06.371754+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:06.371754+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26880,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790082,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:11.707 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:12.064 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:12.064 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:12.077 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-11T01:26:10.672375+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26881,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490200,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625320,"statfs":{"total":289910292480,"available":288384327680,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1225,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002091"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":23604,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:06.371754+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:06.371754+0000","last_peered":"2024-09-11T01:26:06.371754+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:06.371754+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121584+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121584+0000","last_peered":"2024-09-11T01:25:55.121584+0000","last_clean":"2024-09-11T01:25:55.121584+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121584+0000","last_undegraded":"2024-09-11T01:25:55.121584+0000","last_fullsized":"2024-09-11T01:25:55.121584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26880,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790082,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:12.078 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:26:12.078 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 23604, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:06.371754+0000', 'last_change': '2024-09-11T01:25:53.478666+0000', 'last_active': '2024-09-11T01:26:06.371754+0000', 'last_peered': '2024-09-11T01:26:06.371754+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:26:06.371754+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2279, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1728, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:26:12.078 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:12.078 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20492, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118032+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:55.118032+0000', 'last_peered': '2024-09-11T01:25:55.118032+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:55.118032+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118422+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:55.118422+0000', 'last_peered': '2024-09-11T01:25:55.118422+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:55.118422+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20892, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118086+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:55.118086+0000', 'last_peered': '2024-09-11T01:25:55.118086+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:55.118086+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20949, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118255+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:55.118255+0000', 'last_peered': '2024-09-11T01:25:55.118255+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:55.118255+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:12.079 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20918, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118244+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:55.118244+0000', 'last_peered': '2024-09-11T01:25:55.118244+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:55.118244+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:12.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:12.465 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:12.465 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22754/24 objects degraded (94808.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":772,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1525964800,"bytes_avail":288384327680,"bytes_total":289910292480,"degraded_objects":22754,"degraded_total":24,"degraded_ratio":948.08333333333348,"recovering_objects_per_sec":102,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1225,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:12.479 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:12.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:12.788 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:12.788 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:12.802 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-11T01:26:12.672735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27646,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481880,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633640,"statfs":{"total":289910292480,"available":288392847360,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1990,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1990,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002121"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":25134,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:11.373911+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:11.373911+0000","last_peered":"2024-09-11T01:26:11.373911+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:11.373911+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2493,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27645,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790083,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497264,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874576,"statfs":{"total":96636764160,"available":96127565824,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:15.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:16.207 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:16.208 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:16.219 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-11T01:26:14.673070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27646,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481880,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633640,"statfs":{"total":289910292480,"available":288392847360,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002117"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":25134,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:11.373911+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:11.373911+0000","last_peered":"2024-09-11T01:26:11.373911+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:11.373911+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2493,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27645,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790083,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497264,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874576,"statfs":{"total":96636764160,"available":96127565824,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:16.221 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:16.530 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:16.531 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:16.543 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-11T01:26:14.673070+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27646,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481880,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633640,"statfs":{"total":289910292480,"available":288392847360,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1409,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002117"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":25134,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:11.373911+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:11.373911+0000","last_peered":"2024-09-11T01:26:11.373911+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:11.373911+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2493,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21989,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27645,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790083,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497264,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874576,"statfs":{"total":96636764160,"available":96127565824,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:16.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 25134, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:11.373911+0000', 'last_change': '2024-09-11T01:25:53.478666+0000', 'last_active': '2024-09-11T01:26:11.373911+0000', 'last_peered': '2024-09-11T01:26:11.373911+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:26:11.373911+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1514, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2493, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20492, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118032+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:25:55.118032+0000', 'last_peered': '2024-09-11T01:25:55.118032+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:25:55.118032+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21118, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118422+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:25:55.118422+0000', 'last_peered': '2024-09-11T01:25:55.118422+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:25:55.118422+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20892, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118086+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:25:55.118086+0000', 'last_peered': '2024-09-11T01:25:55.118086+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:25:55.118086+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:16.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:16.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20949, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118255+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:25:55.118255+0000', 'last_peered': '2024-09-11T01:25:55.118255+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:25:55.118255+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:16.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:16.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20918, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:55.118244+0000', 'last_change': '2024-09-11T01:15:20.528178+0000', 'last_active': '2024-09-11T01:25:55.118244+0000', 'last_peered': '2024-09-11T01:25:55.118244+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:25:55.118244+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4067, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4065}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:16.546 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:16.940 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:16.940 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21016/24 objects degraded (87566.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":777,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1517445120,"bytes_avail":288392847360,"bytes_total":289910292480,"degraded_objects":21016,"degraded_total":24,"degraded_ratio":875.66666666666652,"recovering_objects_per_sec":198,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2382,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:16.954 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:16.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:17.275 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:17.276 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:17.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-11T01:26:16.673471+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28619,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481880,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281633640,"statfs":{"total":289910292480,"available":288392847360,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2382,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2382,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002092"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28584,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120223+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:25:55.120223+0000","last_peered":"2024-09-11T01:25:55.120223+0000","last_clean":"2024-09-11T01:25:55.120223+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:25:55.120223+0000","last_undegraded":"2024-09-11T01:25:55.120223+0000","last_fullsized":"2024-09-11T01:25:55.120223+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":27080,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:16.368845+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:16.368845+0000","last_peered":"2024-09-11T01:26:16.368845+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:16.368845+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20417,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.118046+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:25:55.118046+0000","last_peered":"2024-09-11T01:25:55.118046+0000","last_clean":"2024-09-11T01:25:55.118046+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:25:55.118046+0000","last_undegraded":"2024-09-11T01:25:55.118046+0000","last_fullsized":"2024-09-11T01:25:55.118046+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20492,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118032+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:25:55.118032+0000","last_peered":"2024-09-11T01:25:55.118032+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:25:55.118032+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118422+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:25:55.118422+0000","last_peered":"2024-09-11T01:25:55.118422+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:25:55.118422+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20562,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117846+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:25:55.117846+0000","last_peered":"2024-09-11T01:25:55.117846+0000","last_clean":"2024-09-11T01:25:55.117846+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:25:55.117846+0000","last_undegraded":"2024-09-11T01:25:55.117846+0000","last_fullsized":"2024-09-11T01:25:55.117846+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20892,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118086+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:25:55.118086+0000","last_peered":"2024-09-11T01:25:55.118086+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:25:55.118086+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.117978+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:25:55.117978+0000","last_peered":"2024-09-11T01:25:55.117978+0000","last_clean":"2024-09-11T01:25:55.117978+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:25:55.117978+0000","last_undegraded":"2024-09-11T01:25:55.117978+0000","last_fullsized":"2024-09-11T01:25:55.117978+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118255+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:25:55.118255+0000","last_peered":"2024-09-11T01:25:55.118255+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:25:55.118255+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:55.118244+0000","last_change":"2024-09-11T01:15:20.528178+0000","last_active":"2024-09-11T01:25:55.118244+0000","last_peered":"2024-09-11T01:25:55.118244+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:25:55.118244+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4065}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21016,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28618,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790083,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497264,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874576,"statfs":{"total":96636764160,"available":96127565824,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:20.290 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:20.614 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:20.615 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:20.629 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-11T01:26:19.810809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28659,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490200,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625320,"statfs":{"total":289910292480,"available":288384327680,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1778,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1778,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.138433"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28586,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.814373+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:18.814373+0000","last_peered":"2024-09-11T01:26:18.814373+0000","last_clean":"2024-09-11T01:26:18.814373+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:18.814373+0000","last_undegraded":"2024-09-11T01:26:18.814373+0000","last_fullsized":"2024-09-11T01:26:18.814373+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":27080,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:16.368845+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:16.368845+0000","last_peered":"2024-09-11T01:26:16.368845+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:16.368845+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20419,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.813083+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:18.813083+0000","last_peered":"2024-09-11T01:26:18.813083+0000","last_clean":"2024-09-11T01:26:18.813083+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:18.813083+0000","last_undegraded":"2024-09-11T01:26:18.813083+0000","last_fullsized":"2024-09-11T01:26:18.813083+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.813002+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:18.813002+0000","last_peered":"2024-09-11T01:26:18.813002+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:18.813002+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21120,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.811659+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:18.811659+0000","last_peered":"2024-09-11T01:26:18.811659+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:18.811659+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20564,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.811422+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:18.811422+0000","last_peered":"2024-09-11T01:26:18.811422+0000","last_clean":"2024-09-11T01:26:18.811422+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:18.811422+0000","last_undegraded":"2024-09-11T01:26:18.811422+0000","last_fullsized":"2024-09-11T01:26:18.811422+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20894,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.814827+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:18.814827+0000","last_peered":"2024-09-11T01:26:18.814827+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:18.814827+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.815293+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:18.815293+0000","last_peered":"2024-09-11T01:26:18.815293+0000","last_clean":"2024-09-11T01:26:18.815293+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:18.815293+0000","last_undegraded":"2024-09-11T01:26:18.815293+0000","last_fullsized":"2024-09-11T01:26:18.815293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20951,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.813454+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:18.813454+0000","last_peered":"2024-09-11T01:26:18.813454+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:18.813454+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":21001,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.070789+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:19.070789+0000","last_peered":"2024-09-11T01:26:19.070789+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:19.070789+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28658,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790084,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:20.630 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:20.957 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:20.957 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:20.970 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-11T01:26:20.674231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28659,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490200,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625320,"statfs":{"total":289910292480,"available":288384327680,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1013,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001496"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28586,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.814373+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:18.814373+0000","last_peered":"2024-09-11T01:26:18.814373+0000","last_clean":"2024-09-11T01:26:18.814373+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:18.814373+0000","last_undegraded":"2024-09-11T01:26:18.814373+0000","last_fullsized":"2024-09-11T01:26:18.814373+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":27080,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:16.368845+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:16.368845+0000","last_peered":"2024-09-11T01:26:16.368845+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:16.368845+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20419,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.813083+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:18.813083+0000","last_peered":"2024-09-11T01:26:18.813083+0000","last_clean":"2024-09-11T01:26:18.813083+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:18.813083+0000","last_undegraded":"2024-09-11T01:26:18.813083+0000","last_fullsized":"2024-09-11T01:26:18.813083+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.813002+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:18.813002+0000","last_peered":"2024-09-11T01:26:18.813002+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:18.813002+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21120,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.811659+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:18.811659+0000","last_peered":"2024-09-11T01:26:18.811659+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:18.811659+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20564,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.811422+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:18.811422+0000","last_peered":"2024-09-11T01:26:18.811422+0000","last_clean":"2024-09-11T01:26:18.811422+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:18.811422+0000","last_undegraded":"2024-09-11T01:26:18.811422+0000","last_fullsized":"2024-09-11T01:26:18.811422+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20894,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.814827+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:18.814827+0000","last_peered":"2024-09-11T01:26:18.814827+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:18.814827+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.815293+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:18.815293+0000","last_peered":"2024-09-11T01:26:18.815293+0000","last_clean":"2024-09-11T01:26:18.815293+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:18.815293+0000","last_undegraded":"2024-09-11T01:26:18.815293+0000","last_fullsized":"2024-09-11T01:26:18.815293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20951,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.813454+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:18.813454+0000","last_peered":"2024-09-11T01:26:18.813454+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:18.813454+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":21001,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.070789+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:19.070789+0000","last_peered":"2024-09-11T01:26:19.070789+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:19.070789+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28658,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790084,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:20.971 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'20046", 'reported_seq': 27080, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:16.368845+0000', 'last_change': '2024-09-11T01:25:53.478666+0000', 'last_active': '2024-09-11T01:26:16.368845+0000', 'last_peered': '2024-09-11T01:26:16.368845+0000', 'last_clean': '2024-09-11T01:15:17.960620+0000', 'last_became_active': '2024-09-11T01:15:20.517177+0000', 'last_became_peered': '2024-09-11T01:15:20.517177+0000', 'last_unstale': '2024-09-11T01:26:16.368845+0000', 'last_undegraded': '2024-09-11T01:15:20.029211+0000', 'last_fullsized': '2024-09-11T01:15:19.987365+0000', 'mapping_epoch': 38, 'log_start': "41'10000", 'ondisk_log_start': "41'10000", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'1981", 'last_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:14:50.130952+0000', 'objects_scrubbed': 1935, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 841, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 541, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18107, 'num_write_kb': 32341, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3466, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -541}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20494, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:18.813002+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:18.813002+0000', 'last_peered': '2024-09-11T01:26:18.813002+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:18.813002+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21120, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:18.811659+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:18.811659+0000', 'last_peered': '2024-09-11T01:26:18.811659+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:18.811659+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20894, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:18.814827+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:18.814827+0000', 'last_peered': '2024-09-11T01:26:18.814827+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:18.814827+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:20.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20951, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:18.813454+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:18.813454+0000', 'last_peered': '2024-09-11T01:26:18.813454+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:18.813454+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:20.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:20.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 21001, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.070789+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:19.070789+0000', 'last_peered': '2024-09-11T01:26:19.070789+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:19.070789+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4027, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 123, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4027}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:20.973 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:21.366 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:21.366 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20976/24 objects degraded (87400.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":781,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1525964800,"bytes_avail":288384327680,"bytes_total":289910292480,"degraded_objects":20976,"degraded_total":24,"degraded_ratio":874,"recovering_objects_per_sec":126,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1013,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:21.380 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:21.380 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:21.728 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:21.728 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:21.740 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-11T01:26:20.674231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28659,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490200,"kb_used_data":199832,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281625320,"statfs":{"total":289910292480,"available":288384327680,"internally_reserved":0,"allocated":204627968,"data_stored":203641569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1013,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1013,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001496"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28586,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.814373+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:18.814373+0000","last_peered":"2024-09-11T01:26:18.814373+0000","last_clean":"2024-09-11T01:26:18.814373+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:18.814373+0000","last_undegraded":"2024-09-11T01:26:18.814373+0000","last_fullsized":"2024-09-11T01:26:18.814373+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20621,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121131+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:25:55.121131+0000","last_peered":"2024-09-11T01:25:55.121131+0000","last_clean":"2024-09-11T01:25:55.121131+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:25:55.121131+0000","last_undegraded":"2024-09-11T01:25:55.121131+0000","last_fullsized":"2024-09-11T01:25:55.121131+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":27080,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:16.368845+0000","last_change":"2024-09-11T01:25:53.478666+0000","last_active":"2024-09-11T01:26:16.368845+0000","last_peered":"2024-09-11T01:26:16.368845+0000","last_clean":"2024-09-11T01:15:17.960620+0000","last_became_active":"2024-09-11T01:15:20.517177+0000","last_became_peered":"2024-09-11T01:15:20.517177+0000","last_unstale":"2024-09-11T01:26:16.368845+0000","last_undegraded":"2024-09-11T01:15:20.029211+0000","last_fullsized":"2024-09-11T01:15:19.987365+0000","mapping_epoch":38,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3466,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20426,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121238+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:25:55.121238+0000","last_peered":"2024-09-11T01:25:55.121238+0000","last_clean":"2024-09-11T01:25:55.121238+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:25:55.121238+0000","last_undegraded":"2024-09-11T01:25:55.121238+0000","last_fullsized":"2024-09-11T01:25:55.121238+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20419,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.813083+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:18.813083+0000","last_peered":"2024-09-11T01:26:18.813083+0000","last_clean":"2024-09-11T01:26:18.813083+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:18.813083+0000","last_undegraded":"2024-09-11T01:26:18.813083+0000","last_fullsized":"2024-09-11T01:26:18.813083+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.813002+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:18.813002+0000","last_peered":"2024-09-11T01:26:18.813002+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:18.813002+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21120,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.811659+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:18.811659+0000","last_peered":"2024-09-11T01:26:18.811659+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:18.811659+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20564,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.811422+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:18.811422+0000","last_peered":"2024-09-11T01:26:18.811422+0000","last_clean":"2024-09-11T01:26:18.811422+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:18.811422+0000","last_undegraded":"2024-09-11T01:26:18.811422+0000","last_fullsized":"2024-09-11T01:26:18.811422+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121505+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:25:55.121505+0000","last_peered":"2024-09-11T01:25:55.121505+0000","last_clean":"2024-09-11T01:25:55.121505+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:25:55.121505+0000","last_undegraded":"2024-09-11T01:25:55.121505+0000","last_fullsized":"2024-09-11T01:25:55.121505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28611,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120909+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:25:55.120909+0000","last_peered":"2024-09-11T01:25:55.120909+0000","last_clean":"2024-09-11T01:25:55.120909+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:25:55.120909+0000","last_undegraded":"2024-09-11T01:25:55.120909+0000","last_fullsized":"2024-09-11T01:25:55.120909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20894,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.814827+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:18.814827+0000","last_peered":"2024-09-11T01:26:18.814827+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:18.814827+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":118,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:26:18.815293+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:18.815293+0000","last_peered":"2024-09-11T01:26:18.815293+0000","last_clean":"2024-09-11T01:26:18.815293+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:18.815293+0000","last_undegraded":"2024-09-11T01:26:18.815293+0000","last_fullsized":"2024-09-11T01:26:18.815293+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28315,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120994+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:25:55.120994+0000","last_peered":"2024-09-11T01:25:55.120994+0000","last_clean":"2024-09-11T01:25:55.120994+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:25:55.120994+0000","last_undegraded":"2024-09-11T01:25:55.120994+0000","last_fullsized":"2024-09-11T01:25:55.120994+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122034+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:25:55.122034+0000","last_peered":"2024-09-11T01:25:55.122034+0000","last_clean":"2024-09-11T01:25:55.122034+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:25:55.122034+0000","last_undegraded":"2024-09-11T01:25:55.122034+0000","last_fullsized":"2024-09-11T01:25:55.122034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29040,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.120706+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:25:55.120706+0000","last_peered":"2024-09-11T01:25:55.120706+0000","last_clean":"2024-09-11T01:25:55.120706+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:25:55.120706+0000","last_undegraded":"2024-09-11T01:25:55.120706+0000","last_fullsized":"2024-09-11T01:25:55.120706+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28841,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121636+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:25:55.121636+0000","last_peered":"2024-09-11T01:25:55.121636+0000","last_clean":"2024-09-11T01:25:55.121636+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:25:55.121636+0000","last_undegraded":"2024-09-11T01:25:55.121636+0000","last_fullsized":"2024-09-11T01:25:55.121636+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122484+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:25:55.122484+0000","last_peered":"2024-09-11T01:25:55.122484+0000","last_clean":"2024-09-11T01:25:55.122484+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:25:55.122484+0000","last_undegraded":"2024-09-11T01:25:55.122484+0000","last_fullsized":"2024-09-11T01:25:55.122484+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20951,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.813454+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:18.813454+0000","last_peered":"2024-09-11T01:26:18.813454+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:18.813454+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121595+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:25:55.121595+0000","last_peered":"2024-09-11T01:25:55.121595+0000","last_clean":"2024-09-11T01:25:55.121595+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:25:55.121595+0000","last_undegraded":"2024-09-11T01:25:55.121595+0000","last_fullsized":"2024-09-11T01:25:55.121595+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":21001,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.070789+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:19.070789+0000","last_peered":"2024-09-11T01:26:19.070789+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:19.070789+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.122090+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:25:55.122090+0000","last_peered":"2024-09-11T01:25:55.122090+0000","last_clean":"2024-09-11T01:25:55.122090+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:25:55.122090+0000","last_undegraded":"2024-09-11T01:25:55.122090+0000","last_fullsized":"2024-09-11T01:25:55.122090+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.121627+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:25:55.121627+0000","last_peered":"2024-09-11T01:25:55.121627+0000","last_clean":"2024-09-11T01:25:55.121627+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:25:55.121627+0000","last_undegraded":"2024-09-11T01:25:55.121627+0000","last_fullsized":"2024-09-11T01:25:55.121627+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20394,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:25:55.266088+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:25:55.266088+0000","last_peered":"2024-09-11T01:25:55.266088+0000","last_clean":"2024-09-11T01:25:55.266088+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:25:55.266088+0000","last_undegraded":"2024-09-11T01:25:55.266088+0000","last_fullsized":"2024-09-11T01:25:55.266088+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20976,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28658,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":203341824,"data_stored":203215369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476252,"kb_used_data":133572,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93895588,"statfs":{"total":96636764160,"available":96149082112,"internally_reserved":0,"allocated":136777728,"data_stored":136457668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790084,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505584,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866256,"statfs":{"total":96636764160,"available":96119046144,"internally_reserved":0,"allocated":491520,"data_stored":149318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508364,"kb_used_data":65780,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863476,"statfs":{"total":96636764160,"available":96116199424,"internally_reserved":0,"allocated":67358720,"data_stored":67034583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":136351744,"data_stored":136315614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:24.742 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:25.092 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:25.093 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:25.105 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-11T01:26:24.674844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29705,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444628,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281670892,"statfs":{"total":289910292480,"available":288430993408,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1325517831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001372"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813371+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813371+0000","last_peered":"2024-09-11T01:26:19.813371+0000","last_clean":"2024-09-11T01:26:19.813371+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813371+0000","last_undegraded":"2024-09-11T01:26:19.813371+0000","last_fullsized":"2024-09-11T01:26:19.813371+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":22013,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:24.075718+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:24.075718+0000","last_peered":"2024-09-11T01:26:24.075718+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:24.075718+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29704,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790085,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:25.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:25.424 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:25.424 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:25.434 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-11T01:26:24.674844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29705,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444628,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281670892,"statfs":{"total":289910292480,"available":288430993408,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1325517831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001372"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813371+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813371+0000","last_peered":"2024-09-11T01:26:19.813371+0000","last_clean":"2024-09-11T01:26:19.813371+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813371+0000","last_undegraded":"2024-09-11T01:26:19.813371+0000","last_fullsized":"2024-09-11T01:26:19.813371+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":22013,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:24.075718+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:24.075718+0000","last_peered":"2024-09-11T01:26:24.075718+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:24.075718+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29704,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790085,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:25.435 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:25.435 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20496, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.812228+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:19.812228+0000', 'last_peered': '2024-09-11T01:26:19.812228+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:19.812228+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813174+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:19.813174+0000', 'last_peered': '2024-09-11T01:26:19.813174+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:19.813174+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20896, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813005+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:19.813005+0000', 'last_peered': '2024-09-11T01:26:19.813005+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:19.813005+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20953, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.814768+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:19.814768+0000', 'last_peered': '2024-09-11T01:26:19.814768+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:19.814768+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:25.436 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 22013, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:24.075718+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:24.075718+0000', 'last_peered': '2024-09-11T01:26:24.075718+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:24.075718+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3522, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 628, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3522}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:25.437 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:25.836 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:25.836 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19930/24 objects degraded (83041.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":786,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1479299072,"bytes_avail":288430993408,"bytes_total":289910292480,"degraded_objects":19930,"degraded_total":24,"degraded_ratio":830.41666666666652,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:25.850 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:25.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:26.185 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:26.186 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:26.199 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-11T01:26:24.674844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29705,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444628,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281670892,"statfs":{"total":289910292480,"available":288430993408,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1325517831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001372"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29044,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813371+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813371+0000","last_peered":"2024-09-11T01:26:19.813371+0000","last_clean":"2024-09-11T01:26:19.813371+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813371+0000","last_undegraded":"2024-09-11T01:26:19.813371+0000","last_fullsized":"2024-09-11T01:26:19.813371+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":22013,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:24.075718+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:24.075718+0000","last_peered":"2024-09-11T01:26:24.075718+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:24.075718+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29704,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790085,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:29.200 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:29.553 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:29.554 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:29.566 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-11T01:26:28.675512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29705,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.865364"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":22013,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:24.075718+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:24.075718+0000","last_peered":"2024-09-11T01:26:24.075718+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:24.075718+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29704,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790086,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:29.567 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:29.884 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:29.885 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:29.898 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-11T01:26:28.675512+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29705,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1086,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.865364"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":22013,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:24.075718+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:24.075718+0000","last_peered":"2024-09-11T01:26:24.075718+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:24.075718+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3522,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":628,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19930,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29704,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790086,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:29.899 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:29.899 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20496, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.812228+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:19.812228+0000', 'last_peered': '2024-09-11T01:26:19.812228+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:19.812228+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:29.899 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813174+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:19.813174+0000', 'last_peered': '2024-09-11T01:26:19.813174+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:19.813174+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20896, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813005+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:19.813005+0000', 'last_peered': '2024-09-11T01:26:19.813005+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:19.813005+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20953, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.814768+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:19.814768+0000', 'last_peered': '2024-09-11T01:26:19.814768+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:19.814768+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:29.900 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 22013, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:24.075718+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:24.075718+0000', 'last_peered': '2024-09-11T01:26:24.075718+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:24.075718+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3522, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 628, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3522}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:29.900 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:30.383 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:30.383 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19930/24 objects degraded (83041.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":790,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1466519552,"bytes_avail":288443772928,"bytes_total":289910292480,"degraded_objects":19930,"degraded_total":24,"degraded_ratio":830.41666666666652,"recovering_objects_per_sec":110,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1086,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:30.396 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:30.396 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:30.750 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:30.750 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:30.765 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-09-11T01:26:30.675849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30274,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.865039"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":23151,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:29.080031+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:29.080031+0000","last_peered":"2024-09-11T01:26:29.080031+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:29.080031+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30273,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790086,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:33.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:34.078 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:34.078 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:34.091 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-11T01:26:32.676197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30274,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1436308,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281679212,"statfs":{"total":289910292480,"available":288439513088,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001966"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":23151,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:29.080031+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:29.080031+0000","last_peered":"2024-09-11T01:26:29.080031+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:29.080031+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30273,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790087,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:34.092 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:34.477 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:34.478 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:34.491 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-11T01:26:32.676197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30274,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1436308,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281679212,"statfs":{"total":289910292480,"available":288439513088,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001966"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":23151,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:29.080031+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:29.080031+0000","last_peered":"2024-09-11T01:26:29.080031+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:29.080031+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2953}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30273,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790087,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20496, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.812228+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:19.812228+0000', 'last_peered': '2024-09-11T01:26:19.812228+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:19.812228+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813174+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:19.813174+0000', 'last_peered': '2024-09-11T01:26:19.813174+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:19.813174+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20896, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813005+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:19.813005+0000', 'last_peered': '2024-09-11T01:26:19.813005+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:19.813005+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:34.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20953, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.814768+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:19.814768+0000', 'last_peered': '2024-09-11T01:26:19.814768+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:19.814768+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:34.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:34.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 23151, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:29.080031+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:29.080031+0000', 'last_peered': '2024-09-11T01:26:29.080031+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:29.080031+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1197, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2953}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:34.494 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:34.884 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:34.885 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19361/24 objects degraded (80670.833%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":795,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1470779392,"bytes_avail":288439513088,"bytes_total":289910292480,"degraded_objects":19361,"degraded_total":24,"degraded_ratio":806.70833333333348,"recovering_objects_per_sec":134,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1615,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:34.898 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:34.899 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:35.215 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:35.216 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:35.229 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":444,"stamp":"2024-09-11T01:26:34.676490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30880,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1436308,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281679212,"statfs":{"total":289910292480,"available":288439513088,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1680,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1680,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001979"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":24363,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:34.079454+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:34.079454+0000","last_peered":"2024-09-11T01:26:34.079454+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:34.079454+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30879,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790087,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:38.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:38.550 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:38.551 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:38.569 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-11T01:26:36.676811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30880,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1436308,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281679212,"statfs":{"total":289910292480,"available":288439513088,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1316998151},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001967"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":24363,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:34.079454+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:34.079454+0000","last_peered":"2024-09-11T01:26:34.079454+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:34.079454+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30879,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790087,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:38.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:38.901 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:38.901 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:38.915 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-11T01:26:38.677139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30880,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444628,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281670892,"statfs":{"total":289910292480,"available":288430993408,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1325517831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001971"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":24363,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:34.079454+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:34.079454+0000","last_peered":"2024-09-11T01:26:34.079454+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:34.079454+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30879,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790088,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:38.916 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:38.916 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20496, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.812228+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:19.812228+0000', 'last_peered': '2024-09-11T01:26:19.812228+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:19.812228+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:38.916 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:38.916 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813174+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:19.813174+0000', 'last_peered': '2024-09-11T01:26:19.813174+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:19.813174+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:38.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:38.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20896, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813005+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:19.813005+0000', 'last_peered': '2024-09-11T01:26:19.813005+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:19.813005+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:38.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:38.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20953, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.814768+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:19.814768+0000', 'last_peered': '2024-09-11T01:26:19.814768+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:19.814768+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:38.917 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:38.917 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 24363, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:34.079454+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:34.079454+0000', 'last_peered': '2024-09-11T01:26:34.079454+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:34.079454+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2347, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1803, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2347}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:38.917 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:39.299 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:39.299 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18755/24 objects degraded (78145.833%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":799,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1479299072,"bytes_avail":288430993408,"bytes_total":289910292480,"degraded_objects":18755,"degraded_total":24,"degraded_ratio":781.45833333333348,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1175,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:39.312 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:39.313 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:39.621 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:39.621 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:39.633 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-09-11T01:26:38.677139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30880,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1444628,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281670892,"statfs":{"total":289910292480,"available":288430993408,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1325517831},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001971"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":24363,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:34.079454+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:34.079454+0000","last_peered":"2024-09-11T01:26:34.079454+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:34.079454+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1803,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18755,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30879,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790088,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:42.635 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:42.944 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:42.944 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:42.957 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-11T01:26:42.677928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31591,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002078"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":25785,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:39.078037+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:39.078037+0000","last_peered":"2024-09-11T01:26:39.078037+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:39.078037+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2514,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31590,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790089,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:42.958 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:43.271 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:43.271 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:43.284 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-11T01:26:42.677928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31591,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002078"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":25785,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:39.078037+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:39.078037+0000","last_peered":"2024-09-11T01:26:39.078037+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:39.078037+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2514,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31590,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790089,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20496, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.812228+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:19.812228+0000', 'last_peered': '2024-09-11T01:26:19.812228+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:19.812228+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813174+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:19.813174+0000', 'last_peered': '2024-09-11T01:26:19.813174+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:19.813174+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20896, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813005+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:19.813005+0000', 'last_peered': '2024-09-11T01:26:19.813005+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:19.813005+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:43.286 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20953, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.814768+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:19.814768+0000', 'last_peered': '2024-09-11T01:26:19.814768+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:19.814768+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:43.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:43.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 25785, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:39.078037+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:39.078037+0000', 'last_peered': '2024-09-11T01:26:39.078037+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:39.078037+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1636, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2514, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:43.287 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:43.692 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18044/24 objects degraded (75183.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":804,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1466519552,"bytes_avail":288443772928,"bytes_total":289910292480,"degraded_objects":18044,"degraded_total":24,"degraded_ratio":751.83333333333348,"recovering_objects_per_sec":109,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:43.705 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:43.705 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:44.047 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:44.047 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:44.059 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-11T01:26:42.677928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31591,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002078"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812467+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812467+0000","last_peered":"2024-09-11T01:26:19.812467+0000","last_clean":"2024-09-11T01:26:19.812467+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812467+0000","last_undegraded":"2024-09-11T01:26:19.812467+0000","last_fullsized":"2024-09-11T01:26:19.812467+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":25785,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:39.078037+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:39.078037+0000","last_peered":"2024-09-11T01:26:39.078037+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:39.078037+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2514,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31590,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790089,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:47.062 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:47.412 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:47.412 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:47.426 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T01:26:46.981232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32517,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1637,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.304420"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812505+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812505+0000","last_peered":"2024-09-11T01:26:19.812505+0000","last_clean":"2024-09-11T01:26:19.812505+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812505+0000","last_undegraded":"2024-09-11T01:26:19.812505+0000","last_fullsized":"2024-09-11T01:26:19.812505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":27637,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:44.080973+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:44.080973+0000","last_peered":"2024-09-11T01:26:44.080973+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:44.080973+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32516,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790089,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:47.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:47.763 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:47.763 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:47.778 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T01:26:46.981232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32517,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1432148,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281683372,"statfs":{"total":289910292480,"available":288443772928,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1637,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.304420"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20233,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.909647+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:19.909647+0000","last_peered":"2024-09-11T01:26:19.909647+0000","last_clean":"2024-09-11T01:26:19.909647+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:19.909647+0000","last_undegraded":"2024-09-11T01:26:19.909647+0000","last_fullsized":"2024-09-11T01:26:19.909647+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812104+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:19.812104+0000","last_peered":"2024-09-11T01:26:19.812104+0000","last_clean":"2024-09-11T01:26:19.812104+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:19.812104+0000","last_undegraded":"2024-09-11T01:26:19.812104+0000","last_fullsized":"2024-09-11T01:26:19.812104+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28615,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812682+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:19.812682+0000","last_peered":"2024-09-11T01:26:19.812682+0000","last_clean":"2024-09-11T01:26:19.812682+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:19.812682+0000","last_undegraded":"2024-09-11T01:26:19.812682+0000","last_fullsized":"2024-09-11T01:26:19.812682+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812505+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812505+0000","last_peered":"2024-09-11T01:26:19.812505+0000","last_clean":"2024-09-11T01:26:19.812505+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812505+0000","last_undegraded":"2024-09-11T01:26:19.812505+0000","last_fullsized":"2024-09-11T01:26:19.812505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812598+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:19.812598+0000","last_peered":"2024-09-11T01:26:19.812598+0000","last_clean":"2024-09-11T01:26:19.812598+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:19.812598+0000","last_undegraded":"2024-09-11T01:26:19.812598+0000","last_fullsized":"2024-09-11T01:26:19.812598+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28845,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812356+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:19.812356+0000","last_peered":"2024-09-11T01:26:19.812356+0000","last_clean":"2024-09-11T01:26:19.812356+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:19.812356+0000","last_undegraded":"2024-09-11T01:26:19.812356+0000","last_fullsized":"2024-09-11T01:26:19.812356+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812856+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:19.812856+0000","last_peered":"2024-09-11T01:26:19.812856+0000","last_clean":"2024-09-11T01:26:19.812856+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:19.812856+0000","last_undegraded":"2024-09-11T01:26:19.812856+0000","last_fullsized":"2024-09-11T01:26:19.812856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":27637,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:44.080973+0000","last_change":"2024-09-11T01:26:18.594387+0000","last_active":"2024-09-11T01:26:44.080973+0000","last_peered":"2024-09-11T01:26:44.080973+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:44.080973+0000","last_undegraded":"2024-09-11T01:15:20.071780+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":710,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-710}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812934+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:19.812934+0000","last_peered":"2024-09-11T01:26:19.812934+0000","last_clean":"2024-09-11T01:26:19.812934+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:19.812934+0000","last_undegraded":"2024-09-11T01:26:19.812934+0000","last_fullsized":"2024-09-11T01:26:19.812934+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812257+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:19.812257+0000","last_peered":"2024-09-11T01:26:19.812257+0000","last_clean":"2024-09-11T01:26:19.812257+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:19.812257+0000","last_undegraded":"2024-09-11T01:26:19.812257+0000","last_fullsized":"2024-09-11T01:26:19.812257+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20398,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812779+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:19.812779+0000","last_peered":"2024-09-11T01:26:19.812779+0000","last_clean":"2024-09-11T01:26:19.812779+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:19.812779+0000","last_undegraded":"2024-09-11T01:26:19.812779+0000","last_fullsized":"2024-09-11T01:26:19.812779+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17118,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32516,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152371200,"data_stored":152259197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790089,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874568,"statfs":{"total":96636764160,"available":96127557632,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20496, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.812228+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:19.812228+0000', 'last_peered': '2024-09-11T01:26:19.812228+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:19.812228+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21122, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813174+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:19.813174+0000', 'last_peered': '2024-09-11T01:26:19.813174+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:19.813174+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20896, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.813005+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:19.813005+0000', 'last_peered': '2024-09-11T01:26:19.813005+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:19.813005+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 20953, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:19.814768+0000', 'last_change': '2024-09-11T01:15:20.548977+0000', 'last_active': '2024-09-11T01:26:19.814768+0000', 'last_peered': '2024-09-11T01:26:19.814768+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:19.814768+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4120, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:47.780 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:47.781 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 27637, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:44.080973+0000', 'last_change': '2024-09-11T01:26:18.594387+0000', 'last_active': '2024-09-11T01:26:44.080973+0000', 'last_peered': '2024-09-11T01:26:44.080973+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:44.080973+0000', 'last_undegraded': '2024-09-11T01:15:20.071780+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 710, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3440, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -710}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:47.781 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:48.338 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:48.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17118/24 objects degraded (71325.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":808,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1466519552,"bytes_avail":288443772928,"bytes_total":289910292480,"degraded_objects":17118,"degraded_total":24,"degraded_ratio":713.25,"recovering_objects_per_sec":136,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1637,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:48.357 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:48.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:48.717 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:48.717 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:48.732 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-11T01:26:48.679009+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33227,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440468,"kb_used_data":150104,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281675052,"statfs":{"total":289910292480,"available":288435253248,"internally_reserved":0,"allocated":153706496,"data_stored":152702446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1636,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001460"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28588,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814819+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:19.814819+0000","last_peered":"2024-09-11T01:26:19.814819+0000","last_clean":"2024-09-11T01:26:19.814819+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:19.814819+0000","last_undegraded":"2024-09-11T01:26:19.814819+0000","last_fullsized":"2024-09-11T01:26:19.814819+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983054+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:46.983054+0000","last_peered":"2024-09-11T01:26:46.983054+0000","last_clean":"2024-09-11T01:26:46.983054+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:46.983054+0000","last_undegraded":"2024-09-11T01:26:46.983054+0000","last_fullsized":"2024-09-11T01:26:46.983054+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20421,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812900+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:19.812900+0000","last_peered":"2024-09-11T01:26:19.812900+0000","last_clean":"2024-09-11T01:26:19.812900+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:19.812900+0000","last_undegraded":"2024-09-11T01:26:19.812900+0000","last_fullsized":"2024-09-11T01:26:19.812900+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20496,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.812228+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:19.812228+0000","last_peered":"2024-09-11T01:26:19.812228+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:19.812228+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813174+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:19.813174+0000","last_peered":"2024-09-11T01:26:19.813174+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:19.813174+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20566,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812381+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:19.812381+0000","last_peered":"2024-09-11T01:26:19.812381+0000","last_clean":"2024-09-11T01:26:19.812381+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:19.812381+0000","last_undegraded":"2024-09-11T01:26:19.812381+0000","last_fullsized":"2024-09-11T01:26:19.812381+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982684+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:46.982684+0000","last_peered":"2024-09-11T01:26:46.982684+0000","last_clean":"2024-09-11T01:26:46.982684+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:46.982684+0000","last_undegraded":"2024-09-11T01:26:46.982684+0000","last_fullsized":"2024-09-11T01:26:46.982684+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28617,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982770+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:46.982770+0000","last_peered":"2024-09-11T01:26:46.982770+0000","last_clean":"2024-09-11T01:26:46.982770+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:46.982770+0000","last_undegraded":"2024-09-11T01:26:46.982770+0000","last_fullsized":"2024-09-11T01:26:46.982770+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20896,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.813005+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:19.813005+0000","last_peered":"2024-09-11T01:26:19.813005+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:19.813005+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":121,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.812505+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:19.812505+0000","last_peered":"2024-09-11T01:26:19.812505+0000","last_clean":"2024-09-11T01:26:19.812505+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:19.812505+0000","last_undegraded":"2024-09-11T01:26:19.812505+0000","last_fullsized":"2024-09-11T01:26:19.812505+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":109,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982667+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:46.982667+0000","last_peered":"2024-09-11T01:26:46.982667+0000","last_clean":"2024-09-11T01:26:46.982667+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:46.982667+0000","last_undegraded":"2024-09-11T01:26:46.982667+0000","last_fullsized":"2024-09-11T01:26:46.982667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28847,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983314+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:46.983314+0000","last_peered":"2024-09-11T01:26:46.983314+0000","last_clean":"2024-09-11T01:26:46.983314+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:46.983314+0000","last_undegraded":"2024-09-11T01:26:46.983314+0000","last_fullsized":"2024-09-11T01:26:46.983314+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983059+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:46.983059+0000","last_peered":"2024-09-11T01:26:46.983059+0000","last_clean":"2024-09-11T01:26:46.983059+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:46.983059+0000","last_undegraded":"2024-09-11T01:26:46.983059+0000","last_fullsized":"2024-09-11T01:26:46.983059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:19.814768+0000","last_change":"2024-09-11T01:15:20.548977+0000","last_active":"2024-09-11T01:26:19.814768+0000","last_peered":"2024-09-11T01:26:19.814768+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:19.814768+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20909,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-11T01:26:46.800453+0000","last_change":"2024-09-11T01:26:47.522394+0000","last_active":"2024-09-11T01:26:47.522394+0000","last_peered":"2024-09-11T01:26:46.800453+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:46.800453+0000","last_undegraded":"2024-09-11T01:26:46.800453+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983307+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:46.983307+0000","last_peered":"2024-09-11T01:26:46.983307+0000","last_clean":"2024-09-11T01:26:46.983307+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:46.983307+0000","last_undegraded":"2024-09-11T01:26:46.983307+0000","last_fullsized":"2024-09-11T01:26:46.983307+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983229+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:46.983229+0000","last_peered":"2024-09-11T01:26:46.983229+0000","last_clean":"2024-09-11T01:26:46.983229+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:46.983229+0000","last_undegraded":"2024-09-11T01:26:46.983229+0000","last_fullsized":"2024-09-11T01:26:46.983229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20400,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982869+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:46.982869+0000","last_peered":"2024-09-11T01:26:46.982869+0000","last_clean":"2024-09-11T01:26:46.982869+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:46.982869+0000","last_undegraded":"2024-09-11T01:26:46.982869+0000","last_fullsized":"2024-09-11T01:26:46.982869+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16408,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33226,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152358912,"data_stored":152257748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790090,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866248,"statfs":{"total":96636764160,"available":96119037952,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508380,"kb_used_data":65796,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863460,"statfs":{"total":96636764160,"available":96116183040,"internally_reserved":0,"allocated":67375104,"data_stored":67040266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:51.735 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:52.100 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:52.101 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:52.113 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-11T01:26:50.679344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33600,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440480,"kb_used_data":150120,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281675040,"statfs":{"total":289910292480,"available":288435240960,"internally_reserved":0,"allocated":153722880,"data_stored":152708079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2009,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001416"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983054+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:46.983054+0000","last_peered":"2024-09-11T01:26:46.983054+0000","last_clean":"2024-09-11T01:26:46.983054+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:46.983054+0000","last_undegraded":"2024-09-11T01:26:46.983054+0000","last_fullsized":"2024-09-11T01:26:46.983054+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982684+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:46.982684+0000","last_peered":"2024-09-11T01:26:46.982684+0000","last_clean":"2024-09-11T01:26:46.982684+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:46.982684+0000","last_undegraded":"2024-09-11T01:26:46.982684+0000","last_fullsized":"2024-09-11T01:26:46.982684+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28617,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982770+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:46.982770+0000","last_peered":"2024-09-11T01:26:46.982770+0000","last_clean":"2024-09-11T01:26:46.982770+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:46.982770+0000","last_undegraded":"2024-09-11T01:26:46.982770+0000","last_fullsized":"2024-09-11T01:26:46.982770+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":109,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982667+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:46.982667+0000","last_peered":"2024-09-11T01:26:46.982667+0000","last_clean":"2024-09-11T01:26:46.982667+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:46.982667+0000","last_undegraded":"2024-09-11T01:26:46.982667+0000","last_fullsized":"2024-09-11T01:26:46.982667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28847,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983314+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:46.983314+0000","last_peered":"2024-09-11T01:26:46.983314+0000","last_clean":"2024-09-11T01:26:46.983314+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:46.983314+0000","last_undegraded":"2024-09-11T01:26:46.983314+0000","last_fullsized":"2024-09-11T01:26:46.983314+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983059+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:46.983059+0000","last_peered":"2024-09-11T01:26:46.983059+0000","last_clean":"2024-09-11T01:26:46.983059+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:46.983059+0000","last_undegraded":"2024-09-11T01:26:46.983059+0000","last_fullsized":"2024-09-11T01:26:46.983059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":21704,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:49.080833+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:49.080833+0000","last_peered":"2024-09-11T01:26:49.080833+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:49.080833+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20909,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-11T01:26:46.800453+0000","last_change":"2024-09-11T01:26:47.522394+0000","last_active":"2024-09-11T01:26:47.522394+0000","last_peered":"2024-09-11T01:26:46.800453+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:46.800453+0000","last_undegraded":"2024-09-11T01:26:46.800453+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983307+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:46.983307+0000","last_peered":"2024-09-11T01:26:46.983307+0000","last_clean":"2024-09-11T01:26:46.983307+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:46.983307+0000","last_undegraded":"2024-09-11T01:26:46.983307+0000","last_fullsized":"2024-09-11T01:26:46.983307+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983229+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:46.983229+0000","last_peered":"2024-09-11T01:26:46.983229+0000","last_clean":"2024-09-11T01:26:46.983229+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:46.983229+0000","last_undegraded":"2024-09-11T01:26:46.983229+0000","last_fullsized":"2024-09-11T01:26:46.983229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20400,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982869+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:46.982869+0000","last_peered":"2024-09-11T01:26:46.982869+0000","last_clean":"2024-09-11T01:26:46.982869+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:46.982869+0000","last_undegraded":"2024-09-11T01:26:46.982869+0000","last_fullsized":"2024-09-11T01:26:46.982869+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33599,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152358912,"data_stored":152257748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790090,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866248,"statfs":{"total":96636764160,"available":96119037952,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:52.115 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:52.514 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:52.514 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:52.531 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-11T01:26:50.679344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33600,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440480,"kb_used_data":150120,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281675040,"statfs":{"total":289910292480,"available":288435240960,"internally_reserved":0,"allocated":153722880,"data_stored":152708079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1321257991},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2009,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001416"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20625,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814215+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:19.814215+0000","last_peered":"2024-09-11T01:26:19.814215+0000","last_clean":"2024-09-11T01:26:19.814215+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:19.814215+0000","last_undegraded":"2024-09-11T01:26:19.814215+0000","last_fullsized":"2024-09-11T01:26:19.814215+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20235,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983054+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:46.983054+0000","last_peered":"2024-09-11T01:26:46.983054+0000","last_clean":"2024-09-11T01:26:46.983054+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:46.983054+0000","last_undegraded":"2024-09-11T01:26:46.983054+0000","last_fullsized":"2024-09-11T01:26:46.983054+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20430,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813311+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:19.813311+0000","last_peered":"2024-09-11T01:26:19.813311+0000","last_clean":"2024-09-11T01:26:19.813311+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:19.813311+0000","last_undegraded":"2024-09-11T01:26:19.813311+0000","last_fullsized":"2024-09-11T01:26:19.813311+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982684+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:46.982684+0000","last_peered":"2024-09-11T01:26:46.982684+0000","last_clean":"2024-09-11T01:26:46.982684+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:46.982684+0000","last_undegraded":"2024-09-11T01:26:46.982684+0000","last_fullsized":"2024-09-11T01:26:46.982684+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28617,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982770+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:46.982770+0000","last_peered":"2024-09-11T01:26:46.982770+0000","last_clean":"2024-09-11T01:26:46.982770+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:46.982770+0000","last_undegraded":"2024-09-11T01:26:46.982770+0000","last_fullsized":"2024-09-11T01:26:46.982770+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813566+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:19.813566+0000","last_peered":"2024-09-11T01:26:19.813566+0000","last_clean":"2024-09-11T01:26:19.813566+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:19.813566+0000","last_undegraded":"2024-09-11T01:26:19.813566+0000","last_fullsized":"2024-09-11T01:26:19.813566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28319,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814127+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:19.814127+0000","last_peered":"2024-09-11T01:26:19.814127+0000","last_clean":"2024-09-11T01:26:19.814127+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:19.814127+0000","last_undegraded":"2024-09-11T01:26:19.814127+0000","last_fullsized":"2024-09-11T01:26:19.814127+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":109,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982667+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:46.982667+0000","last_peered":"2024-09-11T01:26:46.982667+0000","last_clean":"2024-09-11T01:26:46.982667+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:46.982667+0000","last_undegraded":"2024-09-11T01:26:46.982667+0000","last_fullsized":"2024-09-11T01:26:46.982667+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29045,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.813420+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:19.813420+0000","last_peered":"2024-09-11T01:26:19.813420+0000","last_clean":"2024-09-11T01:26:19.813420+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:19.813420+0000","last_undegraded":"2024-09-11T01:26:19.813420+0000","last_fullsized":"2024-09-11T01:26:19.813420+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28847,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983314+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:46.983314+0000","last_peered":"2024-09-11T01:26:46.983314+0000","last_clean":"2024-09-11T01:26:46.983314+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:46.983314+0000","last_undegraded":"2024-09-11T01:26:46.983314+0000","last_fullsized":"2024-09-11T01:26:46.983314+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983059+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:46.983059+0000","last_peered":"2024-09-11T01:26:46.983059+0000","last_clean":"2024-09-11T01:26:46.983059+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:46.983059+0000","last_undegraded":"2024-09-11T01:26:46.983059+0000","last_fullsized":"2024-09-11T01:26:46.983059+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":21704,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:49.080833+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:49.080833+0000","last_peered":"2024-09-11T01:26:49.080833+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:49.080833+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:26:19.814131+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:19.814131+0000","last_peered":"2024-09-11T01:26:19.814131+0000","last_clean":"2024-09-11T01:26:19.814131+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:19.814131+0000","last_undegraded":"2024-09-11T01:26:19.814131+0000","last_fullsized":"2024-09-11T01:26:19.814131+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20909,"reported_epoch":54,"state":"peering","last_fresh":"2024-09-11T01:26:46.800453+0000","last_change":"2024-09-11T01:26:47.522394+0000","last_active":"2024-09-11T01:26:47.522394+0000","last_peered":"2024-09-11T01:26:46.800453+0000","last_clean":"2024-09-11T01:15:17.959501+0000","last_became_active":"2024-09-11T01:15:20.516959+0000","last_became_peered":"2024-09-11T01:15:20.516959+0000","last_unstale":"2024-09-11T01:26:46.800453+0000","last_undegraded":"2024-09-11T01:26:46.800453+0000","last_fullsized":"2024-09-11T01:15:19.986123+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":99,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983307+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:46.983307+0000","last_peered":"2024-09-11T01:26:46.983307+0000","last_clean":"2024-09-11T01:26:46.983307+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:46.983307+0000","last_undegraded":"2024-09-11T01:26:46.983307+0000","last_fullsized":"2024-09-11T01:26:46.983307+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.983229+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:46.983229+0000","last_peered":"2024-09-11T01:26:46.983229+0000","last_clean":"2024-09-11T01:26:46.983229+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:46.983229+0000","last_undegraded":"2024-09-11T01:26:46.983229+0000","last_fullsized":"2024-09-11T01:26:46.983229+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20400,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-11T01:26:46.982869+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:46.982869+0000","last_peered":"2024-09-11T01:26:46.982869+0000","last_clean":"2024-09-11T01:26:46.982869+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:46.982869+0000","last_undegraded":"2024-09-11T01:26:46.982869+0000","last_fullsized":"2024-09-11T01:26:46.982869+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33599,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152358912,"data_stored":152257748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426496,"kb_used_data":83820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93945344,"statfs":{"total":96636764160,"available":96200032256,"internally_reserved":0,"allocated":85831680,"data_stored":85508145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790090,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866248,"statfs":{"total":96636764160,"available":96119037952,"internally_reserved":0,"allocated":499712,"data_stored":154035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85389312,"data_stored":85360408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:52.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:52.532 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20500, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989273+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:47.989273+0000', 'last_peered': '2024-09-11T01:26:47.989273+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:47.989273+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:52.532 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:52.532 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21126, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989247+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:47.989247+0000', 'last_peered': '2024-09-11T01:26:47.989247+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:47.989247+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:52.533 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:52.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20900, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.990306+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:47.990306+0000', 'last_peered': '2024-09-11T01:26:47.990306+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:47.990306+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:52.533 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:52.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 21704, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:49.080833+0000', 'last_change': '2024-09-11T01:26:46.800792+0000', 'last_active': '2024-09-11T01:26:49.080833+0000', 'last_peered': '2024-09-11T01:26:49.080833+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:49.080833+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3747, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 448, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:52.533 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:26:52.533 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'20600", 'reported_seq': 20909, 'reported_epoch': 54, 'state': 'peering', 'last_fresh': '2024-09-11T01:26:46.800453+0000', 'last_change': '2024-09-11T01:26:47.522394+0000', 'last_active': '2024-09-11T01:26:47.522394+0000', 'last_peered': '2024-09-11T01:26:46.800453+0000', 'last_clean': '2024-09-11T01:15:17.959501+0000', 'last_became_active': '2024-09-11T01:15:20.516959+0000', 'last_became_peered': '2024-09-11T01:15:20.516959+0000', 'last_unstale': '2024-09-11T01:26:46.800453+0000', 'last_undegraded': '2024-09-11T01:26:46.800453+0000', 'last_fullsized': '2024-09-11T01:15:19.986123+0000', 'mapping_epoch': 54, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3518", 'last_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:07.160741+0000', 'objects_scrubbed': 3461, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1304, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17135, 'num_write_kb': 27348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4150, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T01:26:52.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:52.949 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:52.949 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16035/24 objects degraded (66812.500%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":813,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1475051520,"bytes_avail":288435240960,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":16035,"degraded_total":24,"degraded_ratio":668.125,"recovering_objects_per_sec":251,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2009,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:52.962 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:52.962 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:53.275 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:53.276 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:53.288 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":455,"stamp":"2024-09-11T01:26:52.679678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33600,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415592,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281699928,"statfs":{"total":289910292480,"available":288460726272,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1083,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1083,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001412"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987519+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987519+0000","last_peered":"2024-09-11T01:26:47.987519+0000","last_clean":"2024-09-11T01:26:47.987519+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987519+0000","last_undegraded":"2024-09-11T01:26:47.987519+0000","last_fullsized":"2024-09-11T01:26:47.987519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28323,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987610+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987610+0000","last_peered":"2024-09-11T01:26:47.987610+0000","last_clean":"2024-09-11T01:26:47.987610+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987610+0000","last_undegraded":"2024-09-11T01:26:47.987610+0000","last_fullsized":"2024-09-11T01:26:47.987610+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":21704,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:49.080833+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:49.080833+0000","last_peered":"2024-09-11T01:26:49.080833+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:49.080833+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3747,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988079+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988079+0000","last_peered":"2024-09-11T01:26:47.988079+0000","last_clean":"2024-09-11T01:26:47.988079+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988079+0000","last_undegraded":"2024-09-11T01:26:47.988079+0000","last_fullsized":"2024-09-11T01:26:47.988079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33599,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790091,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497276,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874564,"statfs":{"total":96636764160,"available":96127553536,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:56.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:56.603 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:56.604 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:56.617 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-11T01:26:54.680050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34515,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415592,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281699928,"statfs":{"total":289910292480,"available":288460726272,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001395"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987519+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987519+0000","last_peered":"2024-09-11T01:26:47.987519+0000","last_clean":"2024-09-11T01:26:47.987519+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987519+0000","last_undegraded":"2024-09-11T01:26:47.987519+0000","last_fullsized":"2024-09-11T01:26:47.987519+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28323,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987610+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987610+0000","last_peered":"2024-09-11T01:26:47.987610+0000","last_clean":"2024-09-11T01:26:47.987610+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987610+0000","last_undegraded":"2024-09-11T01:26:47.987610+0000","last_fullsized":"2024-09-11T01:26:47.987610+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":23534,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:54.085536+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:54.085536+0000","last_peered":"2024-09-11T01:26:54.085536+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:54.085536+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1363,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2832}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988079+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988079+0000","last_peered":"2024-09-11T01:26:47.988079+0000","last_clean":"2024-09-11T01:26:47.988079+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988079+0000","last_undegraded":"2024-09-11T01:26:47.988079+0000","last_fullsized":"2024-09-11T01:26:47.988079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34514,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790091,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497276,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874564,"statfs":{"total":96636764160,"available":96127553536,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:56.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:56.939 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:56.940 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:56.952 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-11T01:26:56.680446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34515,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415592,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281699928,"statfs":{"total":289910292480,"available":288460726272,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.699214"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":23534,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:54.085536+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:54.085536+0000","last_peered":"2024-09-11T01:26:54.085536+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:54.085536+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1363,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2832}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34514,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790091,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497276,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874564,"statfs":{"total":96636764160,"available":96127553536,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:56.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:26:56.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20500, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989273+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:47.989273+0000', 'last_peered': '2024-09-11T01:26:47.989273+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:47.989273+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:56.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:56.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21126, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989247+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:47.989247+0000', 'last_peered': '2024-09-11T01:26:47.989247+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:47.989247+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:56.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:56.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20900, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.990306+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:47.990306+0000', 'last_peered': '2024-09-11T01:26:47.990306+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:47.990306+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:56.954 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:26:56.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 23534, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:54.085536+0000', 'last_change': '2024-09-11T01:26:46.800792+0000', 'last_active': '2024-09-11T01:26:54.085536+0000', 'last_peered': '2024-09-11T01:26:54.085536+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:54.085536+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2832, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1363, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2832}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:56.954 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:57.338 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:57.338 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15120/24 objects degraded (63000.000%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":817,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1449566208,"bytes_avail":288460726272,"bytes_total":289910292480,"degraded_objects":15120,"degraded_total":24,"degraded_ratio":630,"recovering_objects_per_sec":205,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:57.351 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:57.352 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:57.687 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:26:57.687 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:26:57.701 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-11T01:26:56.680446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34515,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415592,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1281971,"kb_avail":281699928,"statfs":{"total":289910292480,"available":288460726272,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1312738311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.699214"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":23534,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:54.085536+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:54.085536+0000","last_peered":"2024-09-11T01:26:54.085536+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:54.085536+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1363,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2832}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34514,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790091,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497276,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874564,"statfs":{"total":96636764160,"available":96127553536,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:00.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:01.017 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:01.017 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:01.029 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-11T01:27:00.681083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35171,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1441960,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281673560,"statfs":{"total":289910292480,"available":288433725440,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1339739143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002073"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":24846,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:59.082217+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:59.082217+0000","last_peered":"2024-09-11T01:26:59.082217+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:59.082217+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35170,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523644,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848196,"statfs":{"total":96636764160,"available":96100552704,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:01.030 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:01.372 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:01.373 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:01.385 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-11T01:27:00.681083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35171,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1441960,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281673560,"statfs":{"total":289910292480,"available":288433725440,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1339739143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002073"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":24846,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:59.082217+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:59.082217+0000","last_peered":"2024-09-11T01:26:59.082217+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:59.082217+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35170,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523644,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848196,"statfs":{"total":96636764160,"available":96100552704,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20500, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989273+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:47.989273+0000', 'last_peered': '2024-09-11T01:26:47.989273+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:47.989273+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21126, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989247+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:47.989247+0000', 'last_peered': '2024-09-11T01:26:47.989247+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:47.989247+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20900, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.990306+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:47.990306+0000', 'last_peered': '2024-09-11T01:26:47.990306+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:47.990306+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:27:01.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 24846, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:59.082217+0000', 'last_change': '2024-09-11T01:26:46.800792+0000', 'last_active': '2024-09-11T01:26:59.082217+0000', 'last_peered': '2024-09-11T01:26:59.082217+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:26:59.082217+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2176, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2019, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2176}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:01.388 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:01.844 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:01.845 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14464/24 objects degraded (60266.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":822,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1476567040,"bytes_avail":288433725440,"bytes_total":289910292480,"degraded_objects":14464,"degraded_total":24,"degraded_ratio":602.66666666666663,"recovering_objects_per_sec":161,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1944,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:01.858 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:01.859 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:02.181 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:02.181 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:02.193 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-11T01:27:00.681083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35171,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1441960,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281673560,"statfs":{"total":289910292480,"available":288433725440,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1339739143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1944,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002073"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":24846,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:59.082217+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:26:59.082217+0000","last_peered":"2024-09-11T01:26:59.082217+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:26:59.082217+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2176,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2176}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35170,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790092,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523644,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848196,"statfs":{"total":96636764160,"available":96100552704,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:05.195 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:05.527 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:05.527 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:05.540 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-11T01:27:04.681760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35952,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446120,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281669400,"statfs":{"total":289910292480,"available":288429465600,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002081"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":26408,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:04.087118+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:27:04.087118+0000","last_peered":"2024-09-11T01:27:04.087118+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:27:04.087118+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35951,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790093,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:05.541 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:05.862 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:05.862 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:05.875 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-11T01:27:04.681760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35952,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446120,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281669400,"statfs":{"total":289910292480,"available":288429465600,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002081"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":26408,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:04.087118+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:27:04.087118+0000","last_peered":"2024-09-11T01:27:04.087118+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:27:04.087118+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35951,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790093,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:05.876 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:05.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20500, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989273+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:47.989273+0000', 'last_peered': '2024-09-11T01:26:47.989273+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:47.989273+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:05.876 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:05.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21126, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989247+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:47.989247+0000', 'last_peered': '2024-09-11T01:26:47.989247+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:47.989247+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:05.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:05.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20900, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.990306+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:47.990306+0000', 'last_peered': '2024-09-11T01:26:47.990306+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:47.990306+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:05.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:27:05.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 26408, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:04.087118+0000', 'last_change': '2024-09-11T01:26:46.800792+0000', 'last_active': '2024-09-11T01:27:04.087118+0000', 'last_peered': '2024-09-11T01:27:04.087118+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:27:04.087118+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1395, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2800, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1395}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:05.877 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:06.263 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:06.263 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13683/24 objects degraded (57012.500%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":826,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1480826880,"bytes_avail":288429465600,"bytes_total":289910292480,"degraded_objects":13683,"degraded_total":24,"degraded_ratio":570.125,"recovering_objects_per_sec":195,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2352,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:06.276 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:06.276 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:06.582 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:06.583 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:06.595 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-11T01:27:04.681760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35952,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446120,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281669400,"statfs":{"total":289910292480,"available":288429465600,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2352,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2352,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002081"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":26408,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:04.087118+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:27:04.087118+0000","last_peered":"2024-09-11T01:27:04.087118+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:27:04.087118+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35951,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790093,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:09.598 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:09.916 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:09.916 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:09.929 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-11T01:27:08.682493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35952,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1437800,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281677720,"statfs":{"total":289910292480,"available":288437985280,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002046"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":26408,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:04.087118+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:27:04.087118+0000","last_peered":"2024-09-11T01:27:04.087118+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:27:04.087118+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35951,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790094,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519484,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852356,"statfs":{"total":96636764160,"available":96104812544,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:09.931 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:10.234 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:10.234 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:10.247 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-11T01:27:08.682493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35952,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1437800,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281677720,"statfs":{"total":289910292480,"available":288437985280,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002046"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":26408,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:04.087118+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:27:04.087118+0000","last_peered":"2024-09-11T01:27:04.087118+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:27:04.087118+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2800,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35951,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790094,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519484,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852356,"statfs":{"total":96636764160,"available":96104812544,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:10.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:10.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 20500, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989273+0000', 'last_change': '2024-09-11T01:15:20.549777+0000', 'last_active': '2024-09-11T01:26:47.989273+0000', 'last_peered': '2024-09-11T01:26:47.989273+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:26:47.989273+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4050, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:10.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:10.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21126, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.989247+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:26:47.989247+0000', 'last_peered': '2024-09-11T01:26:47.989247+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:26:47.989247+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:10.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:10.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20900, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:47.990306+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:26:47.990306+0000', 'last_peered': '2024-09-11T01:26:47.990306+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:26:47.990306+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:10.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:27:10.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'20646", 'reported_seq': 26408, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:04.087118+0000', 'last_change': '2024-09-11T01:26:46.800792+0000', 'last_active': '2024-09-11T01:27:04.087118+0000', 'last_peered': '2024-09-11T01:27:04.087118+0000', 'last_clean': '2024-09-11T01:15:17.926987+0000', 'last_became_active': '2024-09-11T01:15:20.541263+0000', 'last_became_peered': '2024-09-11T01:15:20.541263+0000', 'last_unstale': '2024-09-11T01:27:04.087118+0000', 'last_undegraded': '2024-09-11T01:15:20.100255+0000', 'last_fullsized': '2024-09-11T01:15:19.986399+0000', 'mapping_epoch': 38, 'log_start': "41'10600", 'ondisk_log_start': "41'10600", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3663", 'last_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:09.244556+0000', 'objects_scrubbed': 3599, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1410, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1395, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17039, 'num_write_kb': 26880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2800, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1395}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:10.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:10.649 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:10.650 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13683/24 objects degraded (57012.500%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":831,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1472307200,"bytes_avail":288437985280,"bytes_total":289910292480,"degraded_objects":13683,"degraded_total":24,"degraded_ratio":570.125,"recovering_objects_per_sec":119,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1437,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:10.663 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:10.663 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:10.993 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:10.994 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:11.007 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-11T01:27:10.748918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37031,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1437800,"kb_used_data":133548,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281677720,"statfs":{"total":289910292480,"available":288437985280,"internally_reserved":0,"allocated":136753152,"data_stored":135727688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.067835"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20629,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987830+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:26:47.987830+0000","last_peered":"2024-09-11T01:26:47.987830+0000","last_clean":"2024-09-11T01:26:47.987830+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:26:47.987830+0000","last_undegraded":"2024-09-11T01:26:47.987830+0000","last_fullsized":"2024-09-11T01:26:47.987830+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20237,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990225+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:26:47.990225+0000","last_peered":"2024-09-11T01:26:47.990225+0000","last_clean":"2024-09-11T01:26:47.990225+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:26:47.990225+0000","last_undegraded":"2024-09-11T01:26:47.990225+0000","last_fullsized":"2024-09-11T01:26:47.990225+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20434,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:26:47.988016+0000","last_peered":"2024-09-11T01:26:47.988016+0000","last_clean":"2024-09-11T01:26:47.988016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:26:47.988016+0000","last_undegraded":"2024-09-11T01:26:47.988016+0000","last_fullsized":"2024-09-11T01:26:47.988016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989856+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:26:47.989856+0000","last_peered":"2024-09-11T01:26:47.989856+0000","last_clean":"2024-09-11T01:26:47.989856+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:26:47.989856+0000","last_undegraded":"2024-09-11T01:26:47.989856+0000","last_fullsized":"2024-09-11T01:26:47.989856+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28619,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990266+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:26:47.990266+0000","last_peered":"2024-09-11T01:26:47.990266+0000","last_clean":"2024-09-11T01:26:47.990266+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:26:47.990266+0000","last_undegraded":"2024-09-11T01:26:47.990266+0000","last_fullsized":"2024-09-11T01:26:47.990266+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987564+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:26:47.987564+0000","last_peered":"2024-09-11T01:26:47.987564+0000","last_clean":"2024-09-11T01:26:47.987564+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:26:47.987564+0000","last_undegraded":"2024-09-11T01:26:47.987564+0000","last_fullsized":"2024-09-11T01:26:47.987564+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28324,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.987649+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:26:47.987649+0000","last_peered":"2024-09-11T01:26:47.987649+0000","last_clean":"2024-09-11T01:26:47.987649+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:26:47.987649+0000","last_undegraded":"2024-09-11T01:26:47.987649+0000","last_fullsized":"2024-09-11T01:26:47.987649+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990162+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:26:47.990162+0000","last_peered":"2024-09-11T01:26:47.990162+0000","last_clean":"2024-09-11T01:26:47.990162+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:26:47.990162+0000","last_undegraded":"2024-09-11T01:26:47.990162+0000","last_fullsized":"2024-09-11T01:26:47.990162+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29049,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988112+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:26:47.988112+0000","last_peered":"2024-09-11T01:26:47.988112+0000","last_clean":"2024-09-11T01:26:47.988112+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:26:47.988112+0000","last_undegraded":"2024-09-11T01:26:47.988112+0000","last_fullsized":"2024-09-11T01:26:47.988112+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28849,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990580+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:26:47.990580+0000","last_peered":"2024-09-11T01:26:47.990580+0000","last_clean":"2024-09-11T01:26:47.990580+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:26:47.990580+0000","last_undegraded":"2024-09-11T01:26:47.990580+0000","last_fullsized":"2024-09-11T01:26:47.990580+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990478+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:26:47.990478+0000","last_peered":"2024-09-11T01:26:47.990478+0000","last_clean":"2024-09-11T01:26:47.990478+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:26:47.990478+0000","last_undegraded":"2024-09-11T01:26:47.990478+0000","last_fullsized":"2024-09-11T01:26:47.990478+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":28566,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:09.087916+0000","last_change":"2024-09-11T01:26:46.800792+0000","last_active":"2024-09-11T01:27:09.087916+0000","last_peered":"2024-09-11T01:27:09.087916+0000","last_clean":"2024-09-11T01:15:17.926987+0000","last_became_active":"2024-09-11T01:15:20.541263+0000","last_became_peered":"2024-09-11T01:15:20.541263+0000","last_unstale":"2024-09-11T01:27:09.087916+0000","last_undegraded":"2024-09-11T01:15:20.100255+0000","last_fullsized":"2024-09-11T01:15:19.986399+0000","mapping_epoch":38,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3879,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.988118+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:26:47.988118+0000","last_peered":"2024-09-11T01:26:47.988118+0000","last_clean":"2024-09-11T01:26:47.988118+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:26:47.988118+0000","last_undegraded":"2024-09-11T01:26:47.988118+0000","last_fullsized":"2024-09-11T01:26:47.988118+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20918,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:48.137556+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:26:48.137556+0000","last_peered":"2024-09-11T01:26:48.137556+0000","last_clean":"2024-09-11T01:26:48.137556+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:26:48.137556+0000","last_undegraded":"2024-09-11T01:26:48.137556+0000","last_fullsized":"2024-09-11T01:26:48.137556+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990036+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:26:47.990036+0000","last_peered":"2024-09-11T01:26:47.990036+0000","last_clean":"2024-09-11T01:26:47.990036+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:26:47.990036+0000","last_undegraded":"2024-09-11T01:26:47.990036+0000","last_fullsized":"2024-09-11T01:26:47.990036+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990494+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:26:47.990494+0000","last_peered":"2024-09-11T01:26:47.990494+0000","last_clean":"2024-09-11T01:26:47.990494+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:26:47.990494+0000","last_undegraded":"2024-09-11T01:26:47.990494+0000","last_fullsized":"2024-09-11T01:26:47.990494+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20402,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990384+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:26:47.990384+0000","last_peered":"2024-09-11T01:26:47.990384+0000","last_clean":"2024-09-11T01:26:47.990384+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:26:47.990384+0000","last_undegraded":"2024-09-11T01:26:47.990384+0000","last_fullsized":"2024-09-11T01:26:47.990384+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37030,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135368704,"data_stored":135267540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409924,"kb_used_data":67244,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961916,"statfs":{"total":96636764160,"available":96217001984,"internally_reserved":0,"allocated":68857856,"data_stored":68523570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790094,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519484,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852356,"statfs":{"total":96636764160,"available":96104812544,"internally_reserved":0,"allocated":503808,"data_stored":158219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:14.009 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:14.367 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:14.367 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:14.380 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-11T01:27:12.683213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37347,"num_bytes_recovered":36906428,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446132,"kb_used_data":133560,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281669388,"statfs":{"total":289910292480,"available":288429453312,"internally_reserved":0,"allocated":136765440,"data_stored":135732809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1395,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1395,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001453"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28592,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.991178+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:26:47.991178+0000","last_peered":"2024-09-11T01:26:47.991178+0000","last_clean":"2024-09-11T01:26:47.991178+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:26:47.991178+0000","last_undegraded":"2024-09-11T01:26:47.991178+0000","last_fullsized":"2024-09-11T01:26:47.991178+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20631,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.754417+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:10.754417+0000","last_peered":"2024-09-11T01:27:10.754417+0000","last_clean":"2024-09-11T01:27:10.754417+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:10.754417+0000","last_undegraded":"2024-09-11T01:27:10.754417+0000","last_fullsized":"2024-09-11T01:27:10.754417+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20436,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753716+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:10.753716+0000","last_peered":"2024-09-11T01:27:10.753716+0000","last_clean":"2024-09-11T01:27:10.753716+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:10.753716+0000","last_undegraded":"2024-09-11T01:27:10.753716+0000","last_fullsized":"2024-09-11T01:27:10.753716+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20425,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.990922+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:26:47.990922+0000","last_peered":"2024-09-11T01:26:47.990922+0000","last_clean":"2024-09-11T01:26:47.990922+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:26:47.990922+0000","last_undegraded":"2024-09-11T01:26:47.990922+0000","last_fullsized":"2024-09-11T01:26:47.990922+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20500,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989273+0000","last_change":"2024-09-11T01:15:20.549777+0000","last_active":"2024-09-11T01:26:47.989273+0000","last_peered":"2024-09-11T01:26:47.989273+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:26:47.989273+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21126,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.989247+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:26:47.989247+0000","last_peered":"2024-09-11T01:26:47.989247+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:26:47.989247+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20570,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989065+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:26:47.989065+0000","last_peered":"2024-09-11T01:26:47.989065+0000","last_clean":"2024-09-11T01:26:47.989065+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:26:47.989065+0000","last_undegraded":"2024-09-11T01:26:47.989065+0000","last_fullsized":"2024-09-11T01:26:47.989065+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762121+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762121+0000","last_peered":"2024-09-11T01:27:11.762121+0000","last_clean":"2024-09-11T01:27:11.762121+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762121+0000","last_undegraded":"2024-09-11T01:27:11.762121+0000","last_fullsized":"2024-09-11T01:27:11.762121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753996+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:10.753996+0000","last_peered":"2024-09-11T01:27:10.753996+0000","last_clean":"2024-09-11T01:27:10.753996+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:10.753996+0000","last_undegraded":"2024-09-11T01:27:10.753996+0000","last_fullsized":"2024-09-11T01:27:10.753996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20900,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:47.990306+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:26:47.990306+0000","last_peered":"2024-09-11T01:26:47.990306+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:26:47.990306+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":125,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:26:47.989183+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:26:47.989183+0000","last_peered":"2024-09-11T01:26:47.989183+0000","last_clean":"2024-09-11T01:26:47.989183+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:26:47.989183+0000","last_undegraded":"2024-09-11T01:26:47.989183+0000","last_fullsized":"2024-09-11T01:26:47.989183+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28326,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.754075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:10.754075+0000","last_peered":"2024-09-11T01:27:10.754075+0000","last_clean":"2024-09-11T01:27:10.754075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:10.754075+0000","last_undegraded":"2024-09-11T01:27:10.754075+0000","last_fullsized":"2024-09-11T01:27:10.754075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29051,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753759+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:10.753759+0000","last_peered":"2024-09-11T01:27:10.753759+0000","last_clean":"2024-09-11T01:27:10.753759+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:10.753759+0000","last_undegraded":"2024-09-11T01:27:10.753759+0000","last_fullsized":"2024-09-11T01:27:10.753759+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28853,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762410+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762410+0000","last_peered":"2024-09-11T01:27:11.762410+0000","last_clean":"2024-09-11T01:27:11.762410+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762410+0000","last_undegraded":"2024-09-11T01:27:11.762410+0000","last_fullsized":"2024-09-11T01:27:11.762410+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753792+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:10.753792+0000","last_peered":"2024-09-11T01:27:10.753792+0000","last_clean":"2024-09-11T01:27:10.753792+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:10.753792+0000","last_undegraded":"2024-09-11T01:27:10.753792+0000","last_fullsized":"2024-09-11T01:27:10.753792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12288,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37346,"num_bytes_recovered":36906409,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135364608,"data_stored":135267057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409928,"kb_used_data":67252,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961912,"statfs":{"total":96636764160,"available":96216997888,"internally_reserved":0,"allocated":68866048,"data_stored":68526372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790095,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527812,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844028,"statfs":{"total":96636764160,"available":96096284672,"internally_reserved":0,"allocated":507904,"data_stored":160538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508392,"kb_used_data":65812,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863448,"statfs":{"total":96636764160,"available":96116170752,"internally_reserved":0,"allocated":67391488,"data_stored":67045899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:14.382 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:14.687 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:14.688 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:14.700 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-11T01:27:14.683508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37812,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446148,"kb_used_data":133576,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281669372,"statfs":{"total":289910292480,"available":288429436928,"internally_reserved":0,"allocated":136781824,"data_stored":135738392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001357"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20631,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.754417+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:10.754417+0000","last_peered":"2024-09-11T01:27:10.754417+0000","last_clean":"2024-09-11T01:27:10.754417+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:10.754417+0000","last_undegraded":"2024-09-11T01:27:10.754417+0000","last_fullsized":"2024-09-11T01:27:10.754417+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20436,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753716+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:10.753716+0000","last_peered":"2024-09-11T01:27:10.753716+0000","last_clean":"2024-09-11T01:27:10.753716+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:10.753716+0000","last_undegraded":"2024-09-11T01:27:10.753716+0000","last_fullsized":"2024-09-11T01:27:10.753716+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":21435,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:14.082912+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:14.082912+0000","last_peered":"2024-09-11T01:27:14.082912+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:14.082912+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762121+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762121+0000","last_peered":"2024-09-11T01:27:11.762121+0000","last_clean":"2024-09-11T01:27:11.762121+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762121+0000","last_undegraded":"2024-09-11T01:27:11.762121+0000","last_fullsized":"2024-09-11T01:27:11.762121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753996+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:10.753996+0000","last_peered":"2024-09-11T01:27:10.753996+0000","last_clean":"2024-09-11T01:27:10.753996+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:10.753996+0000","last_undegraded":"2024-09-11T01:27:10.753996+0000","last_fullsized":"2024-09-11T01:27:10.753996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28326,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.754075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:10.754075+0000","last_peered":"2024-09-11T01:27:10.754075+0000","last_clean":"2024-09-11T01:27:10.754075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:10.754075+0000","last_undegraded":"2024-09-11T01:27:10.754075+0000","last_fullsized":"2024-09-11T01:27:10.754075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29051,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753759+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:10.753759+0000","last_peered":"2024-09-11T01:27:10.753759+0000","last_clean":"2024-09-11T01:27:10.753759+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:10.753759+0000","last_undegraded":"2024-09-11T01:27:10.753759+0000","last_fullsized":"2024-09-11T01:27:10.753759+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28853,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762410+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762410+0000","last_peered":"2024-09-11T01:27:11.762410+0000","last_clean":"2024-09-11T01:27:11.762410+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762410+0000","last_undegraded":"2024-09-11T01:27:11.762410+0000","last_fullsized":"2024-09-11T01:27:11.762410+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753792+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:10.753792+0000","last_peered":"2024-09-11T01:27:10.753792+0000","last_clean":"2024-09-11T01:27:10.753792+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:10.753792+0000","last_undegraded":"2024-09-11T01:27:10.753792+0000","last_fullsized":"2024-09-11T01:27:10.753792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37811,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135364608,"data_stored":135267057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409928,"kb_used_data":67252,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961912,"statfs":{"total":96636764160,"available":96216997888,"internally_reserved":0,"allocated":68866048,"data_stored":68526372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790095,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527812,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844028,"statfs":{"total":96636764160,"available":96096284672,"internally_reserved":0,"allocated":507904,"data_stored":160538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:14.701 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:14.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 21435, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:14.082912+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:14.082912+0000', 'last_peered': '2024-09-11T01:27:14.082912+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:14.082912+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3585, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 531, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:14.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:14.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21130, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759729+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:27:11.759729+0000', 'last_peered': '2024-09-11T01:27:11.759729+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:11.759729+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:14.702 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:14.702 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20904, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759843+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:11.759843+0000', 'last_peered': '2024-09-11T01:27:11.759843+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:11.759843+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:14.702 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:15.149 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:15.149 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11823/24 objects degraded (49262.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":835,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1480855552,"bytes_avail":288429436928,"bytes_total":289910292480,"degraded_objects":11823,"degraded_total":24,"degraded_ratio":492.625,"recovering_objects_per_sec":232,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:15.163 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:15.164 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:15.542 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:15.542 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:15.555 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-11T01:27:14.683508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37812,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1446148,"kb_used_data":133576,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281669372,"statfs":{"total":289910292480,"available":288429436928,"internally_reserved":0,"allocated":136781824,"data_stored":135738392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1860,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001357"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20631,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.754417+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:10.754417+0000","last_peered":"2024-09-11T01:27:10.754417+0000","last_clean":"2024-09-11T01:27:10.754417+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:10.754417+0000","last_undegraded":"2024-09-11T01:27:10.754417+0000","last_fullsized":"2024-09-11T01:27:10.754417+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20436,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753716+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:10.753716+0000","last_peered":"2024-09-11T01:27:10.753716+0000","last_clean":"2024-09-11T01:27:10.753716+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:10.753716+0000","last_undegraded":"2024-09-11T01:27:10.753716+0000","last_fullsized":"2024-09-11T01:27:10.753716+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":21435,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:14.082912+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:14.082912+0000","last_peered":"2024-09-11T01:27:14.082912+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:14.082912+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762121+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762121+0000","last_peered":"2024-09-11T01:27:11.762121+0000","last_clean":"2024-09-11T01:27:11.762121+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762121+0000","last_undegraded":"2024-09-11T01:27:11.762121+0000","last_fullsized":"2024-09-11T01:27:11.762121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753996+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:10.753996+0000","last_peered":"2024-09-11T01:27:10.753996+0000","last_clean":"2024-09-11T01:27:10.753996+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:10.753996+0000","last_undegraded":"2024-09-11T01:27:10.753996+0000","last_fullsized":"2024-09-11T01:27:10.753996+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28326,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.754075+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:10.754075+0000","last_peered":"2024-09-11T01:27:10.754075+0000","last_clean":"2024-09-11T01:27:10.754075+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:10.754075+0000","last_undegraded":"2024-09-11T01:27:10.754075+0000","last_fullsized":"2024-09-11T01:27:10.754075+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29051,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753759+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:10.753759+0000","last_peered":"2024-09-11T01:27:10.753759+0000","last_clean":"2024-09-11T01:27:10.753759+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:10.753759+0000","last_undegraded":"2024-09-11T01:27:10.753759+0000","last_fullsized":"2024-09-11T01:27:10.753759+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28853,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762410+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762410+0000","last_peered":"2024-09-11T01:27:11.762410+0000","last_clean":"2024-09-11T01:27:11.762410+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762410+0000","last_undegraded":"2024-09-11T01:27:11.762410+0000","last_fullsized":"2024-09-11T01:27:11.762410+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":90,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-11T01:27:10.753792+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:10.753792+0000","last_peered":"2024-09-11T01:27:10.753792+0000","last_clean":"2024-09-11T01:27:10.753792+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:10.753792+0000","last_undegraded":"2024-09-11T01:27:10.753792+0000","last_fullsized":"2024-09-11T01:27:10.753792+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37811,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135364608,"data_stored":135267057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409928,"kb_used_data":67252,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93961912,"statfs":{"total":96636764160,"available":96216997888,"internally_reserved":0,"allocated":68866048,"data_stored":68526372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790095,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527812,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844028,"statfs":{"total":96636764160,"available":96096284672,"internally_reserved":0,"allocated":507904,"data_stored":160538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68399104,"data_stored":68370200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:18.557 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:18.925 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:18.925 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:18.938 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-11T01:27:18.684284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37812,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":781,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001465"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":21435,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:14.082912+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:14.082912+0000","last_peered":"2024-09-11T01:27:14.082912+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:14.082912+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762121+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762121+0000","last_peered":"2024-09-11T01:27:11.762121+0000","last_clean":"2024-09-11T01:27:11.762121+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762121+0000","last_undegraded":"2024-09-11T01:27:11.762121+0000","last_fullsized":"2024-09-11T01:27:11.762121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37811,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790096,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:18.939 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:19.278 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:19.279 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:19.291 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-11T01:27:18.684284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37812,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":781,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001465"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":21435,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:14.082912+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:14.082912+0000","last_peered":"2024-09-11T01:27:14.082912+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:14.082912+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762121+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762121+0000","last_peered":"2024-09-11T01:27:11.762121+0000","last_clean":"2024-09-11T01:27:11.762121+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762121+0000","last_undegraded":"2024-09-11T01:27:11.762121+0000","last_fullsized":"2024-09-11T01:27:11.762121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37811,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790096,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:19.292 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:19.293 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 21435, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:14.082912+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:14.082912+0000', 'last_peered': '2024-09-11T01:27:14.082912+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:14.082912+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3585, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 531, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:19.293 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:19.293 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21130, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759729+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:27:11.759729+0000', 'last_peered': '2024-09-11T01:27:11.759729+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:11.759729+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:19.293 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:19.293 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20904, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759843+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:11.759843+0000', 'last_peered': '2024-09-11T01:27:11.759843+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:11.759843+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:19.293 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:19.681 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:19.681 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11823/24 objects degraded (49262.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":840,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1450921984,"bytes_avail":288459370496,"bytes_total":289910292480,"degraded_objects":11823,"degraded_total":24,"degraded_ratio":492.625,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":781,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:19.695 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:19.695 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:20.006 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:20.006 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:20.019 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-11T01:27:18.684284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37812,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":781,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001465"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":21435,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:14.082912+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:14.082912+0000","last_peered":"2024-09-11T01:27:14.082912+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:14.082912+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3585,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":531,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3584}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762121+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762121+0000","last_peered":"2024-09-11T01:27:11.762121+0000","last_clean":"2024-09-11T01:27:11.762121+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762121+0000","last_undegraded":"2024-09-11T01:27:11.762121+0000","last_fullsized":"2024-09-11T01:27:11.762121+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37811,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790096,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:23.021 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:23.358 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:23.359 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:23.371 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-11T01:27:22.684891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38362,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421076,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281694444,"statfs":{"total":289910292480,"available":288455110656,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1331,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.926301"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":22535,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:19.088581+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:19.088581+0000","last_peered":"2024-09-11T01:27:19.088581+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:19.088581+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1081,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3034}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38361,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790097,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519496,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852344,"statfs":{"total":96636764160,"available":96104800256,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:23.372 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:23.694 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:23.695 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:23.707 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-11T01:27:22.684891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38362,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421076,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281694444,"statfs":{"total":289910292480,"available":288455110656,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1331,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.926301"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":22535,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:19.088581+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:19.088581+0000","last_peered":"2024-09-11T01:27:19.088581+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:19.088581+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1081,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3034}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38361,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790097,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519496,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852344,"statfs":{"total":96636764160,"available":96104800256,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:23.708 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:23.708 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 22535, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:19.088581+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:19.088581+0000', 'last_peered': '2024-09-11T01:27:19.088581+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:19.088581+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3035, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1081, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3034}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:23.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:23.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21130, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759729+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:27:11.759729+0000', 'last_peered': '2024-09-11T01:27:11.759729+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:11.759729+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:23.709 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:23.709 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20904, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759843+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:11.759843+0000', 'last_peered': '2024-09-11T01:27:11.759843+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:11.759843+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:23.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:24.102 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:24.102 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11273/24 objects degraded (46970.833%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":844,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1455181824,"bytes_avail":288455110656,"bytes_total":289910292480,"degraded_objects":11273,"degraded_total":24,"degraded_ratio":469.70833333333326,"recovering_objects_per_sec":121,"recovering_bytes_per_sec":44,"recovering_keys_per_sec":0,"num_objects_recovered":1331,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:24.116 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:24.116 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:24.439 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:24.440 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:24.453 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-11T01:27:22.684891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38362,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421076,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281694444,"statfs":{"total":289910292480,"available":288455110656,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1331,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1331,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.926301"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":22535,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:19.088581+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:19.088581+0000","last_peered":"2024-09-11T01:27:19.088581+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:19.088581+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3035,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1081,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3034}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21130,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759729+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.759729+0000","last_peered":"2024-09-11T01:27:11.759729+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.759729+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11273,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38361,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790097,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519496,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852344,"statfs":{"total":96636764160,"available":96104800256,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:27.454 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:27.764 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:27.765 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:27.778 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T01:27:26.685560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38977,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421076,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281694444,"statfs":{"total":289910292480,"available":288455110656,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002051"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":23766,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:24.089978+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:24.089978+0000","last_peered":"2024-09-11T01:27:24.089978+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:24.089978+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38976,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790097,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519496,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852344,"statfs":{"total":96636764160,"available":96104800256,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:27.779 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:28.113 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:28.113 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:28.126 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T01:27:26.685560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38977,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1421076,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281694444,"statfs":{"total":289910292480,"available":288455110656,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002051"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":23766,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:24.089978+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:24.089978+0000","last_peered":"2024-09-11T01:27:24.089978+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:24.089978+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38976,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790097,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519496,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852344,"statfs":{"total":96636764160,"available":96104800256,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:28.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:28.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 23766, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:24.089978+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:24.089978+0000', 'last_peered': '2024-09-11T01:27:24.089978+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:24.089978+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2420, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1696, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:28.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:28.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21131, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.762084+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:27:11.762084+0000', 'last_peered': '2024-09-11T01:27:11.762084+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:11.762084+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:28.127 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:28.127 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20904, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759843+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:11.759843+0000', 'last_peered': '2024-09-11T01:27:11.759843+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:11.759843+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:28.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:28.515 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:28.516 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10658/24 objects degraded (44408.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":848,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1455181824,"bytes_avail":288455110656,"bytes_total":289910292480,"degraded_objects":10658,"degraded_total":24,"degraded_ratio":444.08333333333326,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1165,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:28.530 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:28.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:28.872 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:28.873 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:28.888 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-11T01:27:28.685861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38977,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429396,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281686124,"statfs":{"total":289910292480,"available":288446590976,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1165,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001950"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":23766,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:24.089978+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:24.089978+0000","last_peered":"2024-09-11T01:27:24.089978+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:24.089978+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10658,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38976,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790098,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527816,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844024,"statfs":{"total":96636764160,"available":96096280576,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:31.891 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:32.221 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:32.221 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:32.234 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-11T01:27:30.686181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39704,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429396,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281686124,"statfs":{"total":289910292480,"available":288446590976,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001897"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":25220,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:29.093925+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:29.093925+0000","last_peered":"2024-09-11T01:27:29.093925+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:29.093925+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2423,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39703,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790098,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527816,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844024,"statfs":{"total":96636764160,"available":96096280576,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:32.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:32.603 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:32.603 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:32.615 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-11T01:27:30.686181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39704,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429396,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281686124,"statfs":{"total":289910292480,"available":288446590976,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1892,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001897"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":25220,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:29.093925+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:29.093925+0000","last_peered":"2024-09-11T01:27:29.093925+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:29.093925+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2423,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39703,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790098,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527816,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844024,"statfs":{"total":96636764160,"available":96096280576,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:32.617 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:32.617 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 25220, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:29.093925+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:29.093925+0000', 'last_peered': '2024-09-11T01:27:29.093925+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:29.093925+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1693, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2423, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:32.617 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:32.617 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21131, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.762084+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:27:11.762084+0000', 'last_peered': '2024-09-11T01:27:11.762084+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:11.762084+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:32.617 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:32.617 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20904, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759843+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:11.759843+0000', 'last_peered': '2024-09-11T01:27:11.759843+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:11.759843+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:32.618 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:33.012 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:33.012 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9931/24 objects degraded (41379.167%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":853,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1450921984,"bytes_avail":288459370496,"bytes_total":289910292480,"degraded_objects":9931,"degraded_total":24,"degraded_ratio":413.79166666666674,"recovering_objects_per_sec":111,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1342,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:33.025 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:33.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:33.365 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:33.366 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:33.378 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-11T01:27:32.686589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39704,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1342,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001996"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20429,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759660+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759660+0000","last_peered":"2024-09-11T01:27:11.759660+0000","last_clean":"2024-09-11T01:27:11.759660+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759660+0000","last_undegraded":"2024-09-11T01:27:11.759660+0000","last_fullsized":"2024-09-11T01:27:11.759660+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":25220,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:29.093925+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:29.093925+0000","last_peered":"2024-09-11T01:27:29.093925+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:29.093925+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1693,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2423,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9931,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39703,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:36.381 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:36.709 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:36.709 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:36.721 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-11T01:27:36.687298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40641,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1664,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002087"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20430,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759707+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759707+0000","last_peered":"2024-09-11T01:27:11.759707+0000","last_clean":"2024-09-11T01:27:11.759707+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759707+0000","last_undegraded":"2024-09-11T01:27:11.759707+0000","last_fullsized":"2024-09-11T01:27:11.759707+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":27094,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:34.094491+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:34.094491+0000","last_peered":"2024-09-11T01:27:34.094491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:34.094491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40640,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:36.723 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:37.047 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:37.048 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:37.062 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-11T01:27:36.687298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40641,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1664,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002087"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20430,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759707+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759707+0000","last_peered":"2024-09-11T01:27:11.759707+0000","last_clean":"2024-09-11T01:27:11.759707+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759707+0000","last_undegraded":"2024-09-11T01:27:11.759707+0000","last_fullsized":"2024-09-11T01:27:11.759707+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":27094,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:34.094491+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:34.094491+0000","last_peered":"2024-09-11T01:27:34.094491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:34.094491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40640,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:37.063 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:37.063 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 27094, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:34.094491+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:34.094491+0000', 'last_peered': '2024-09-11T01:27:34.094491+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:34.094491+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 756, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3360, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:37.063 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:37.063 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21131, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.762084+0000', 'last_change': '2024-09-11T01:15:20.555175+0000', 'last_active': '2024-09-11T01:27:11.762084+0000', 'last_peered': '2024-09-11T01:27:11.762084+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:11.762084+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4206, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4204}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:37.063 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:37.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20904, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:11.759843+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:11.759843+0000', 'last_peered': '2024-09-11T01:27:11.759843+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:11.759843+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:37.064 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:37.459 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:37.459 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8994/24 objects degraded (37475.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":857,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1450921984,"bytes_avail":288459370496,"bytes_total":289910292480,"degraded_objects":8994,"degraded_total":24,"degraded_ratio":374.75,"recovering_objects_per_sec":138,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1664,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:37.472 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:37.472 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:37.779 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:37.780 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:37.793 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-11T01:27:36.687298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40641,"num_bytes_recovered":36906911,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1416916,"kb_used_data":116824,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281698604,"statfs":{"total":289910292480,"available":288459370496,"internally_reserved":0,"allocated":119627776,"data_stored":118573522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1664,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1664,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002087"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28596,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.760496+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:11.760496+0000","last_peered":"2024-09-11T01:27:11.760496+0000","last_clean":"2024-09-11T01:27:11.760496+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:11.760496+0000","last_undegraded":"2024-09-11T01:27:11.760496+0000","last_fullsized":"2024-09-11T01:27:11.760496+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20430,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759707+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:11.759707+0000","last_peered":"2024-09-11T01:27:11.759707+0000","last_clean":"2024-09-11T01:27:11.759707+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:11.759707+0000","last_undegraded":"2024-09-11T01:27:11.759707+0000","last_fullsized":"2024-09-11T01:27:11.759707+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":27094,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:34.094491+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:34.094491+0000","last_peered":"2024-09-11T01:27:34.094491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:34.094491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21131,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.762084+0000","last_change":"2024-09-11T01:15:20.555175+0000","last_active":"2024-09-11T01:27:11.762084+0000","last_peered":"2024-09-11T01:27:11.762084+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:11.762084+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4204},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20575,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759591+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:11.759591+0000","last_peered":"2024-09-11T01:27:11.759591+0000","last_clean":"2024-09-11T01:27:11.759591+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:11.759591+0000","last_undegraded":"2024-09-11T01:27:11.759591+0000","last_fullsized":"2024-09-11T01:27:11.759591+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20904,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:11.759843+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:11.759843+0000","last_peered":"2024-09-11T01:27:11.759843+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:11.759843+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.759652+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:11.759652+0000","last_peered":"2024-09-11T01:27:11.759652+0000","last_clean":"2024-09-11T01:27:11.759652+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:11.759652+0000","last_undegraded":"2024-09-11T01:27:11.759652+0000","last_fullsized":"2024-09-11T01:27:11.759652+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8994,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40640,"num_bytes_recovered":36906892,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790099,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515336,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856504,"statfs":{"total":96636764160,"available":96109060096,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508408,"kb_used_data":65828,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863432,"statfs":{"total":96636764160,"available":96116154368,"internally_reserved":0,"allocated":67407872,"data_stored":67051482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:40.795 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:41.116 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:41.116 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:41.129 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-11T01:27:40.688031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40948,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429404,"kb_used_data":116832,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281686116,"statfs":{"total":289910292480,"available":288446582784,"internally_reserved":0,"allocated":119635968,"data_stored":118576299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001442"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28600,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856058+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856058+0000","last_peered":"2024-09-11T01:27:38.856058+0000","last_clean":"2024-09-11T01:27:38.856058+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856058+0000","last_undegraded":"2024-09-11T01:27:38.856058+0000","last_fullsized":"2024-09-11T01:27:38.856058+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":27094,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:34.094491+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:34.094491+0000","last_peered":"2024-09-11T01:27:34.094491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:34.094491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21750,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:39.093437+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:39.093437+0000","last_peered":"2024-09-11T01:27:39.093437+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:39.093437+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20908,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.856324+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.856324+0000","last_peered":"2024-09-11T01:27:38.856324+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.856324+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40947,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527816,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844024,"statfs":{"total":96636764160,"available":96096280576,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508416,"kb_used_data":65836,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863424,"statfs":{"total":96636764160,"available":96116146176,"internally_reserved":0,"allocated":67416064,"data_stored":67054259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:41.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:41.485 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:41.486 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:41.501 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-11T01:27:40.688031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40948,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429404,"kb_used_data":116832,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281686116,"statfs":{"total":289910292480,"available":288446582784,"internally_reserved":0,"allocated":119635968,"data_stored":118576299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001442"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28600,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856058+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856058+0000","last_peered":"2024-09-11T01:27:38.856058+0000","last_clean":"2024-09-11T01:27:38.856058+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856058+0000","last_undegraded":"2024-09-11T01:27:38.856058+0000","last_fullsized":"2024-09-11T01:27:38.856058+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":27094,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:34.094491+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:34.094491+0000","last_peered":"2024-09-11T01:27:34.094491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:34.094491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21750,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:39.093437+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:39.093437+0000","last_peered":"2024-09-11T01:27:39.093437+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:39.093437+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20908,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.856324+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.856324+0000","last_peered":"2024-09-11T01:27:38.856324+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.856324+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40947,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527816,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844024,"statfs":{"total":96636764160,"available":96096280576,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508416,"kb_used_data":65836,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863424,"statfs":{"total":96636764160,"available":96116146176,"internally_reserved":0,"allocated":67416064,"data_stored":67054259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:41.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:27:41.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'20207", 'reported_seq': 27094, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:34.094491+0000', 'last_change': '2024-09-11T01:27:10.304340+0000', 'last_active': '2024-09-11T01:27:34.094491+0000', 'last_peered': '2024-09-11T01:27:34.094491+0000', 'last_clean': '2024-09-11T01:15:17.953735+0000', 'last_became_active': '2024-09-11T01:15:20.541862+0000', 'last_became_peered': '2024-09-11T01:15:20.541862+0000', 'last_unstale': '2024-09-11T01:27:34.094491+0000', 'last_undegraded': '2024-09-11T01:15:20.084964+0000', 'last_fullsized': '2024-09-11T01:15:19.985420+0000', 'mapping_epoch': 38, 'log_start': "41'10200", 'ondisk_log_start': "41'10200", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3559", 'last_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:08.270496+0000', 'objects_scrubbed': 3508, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1435, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 756, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16695, 'num_write_kb': 26376, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3360, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -755}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:41.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:41.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 21750, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:39.093437+0000', 'last_change': '2024-09-11T01:27:36.975917+0000', 'last_active': '2024-09-11T01:27:39.093437+0000', 'last_peered': '2024-09-11T01:27:39.093437+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:39.093437+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3899, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 384, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3897}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:41.502 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:41.502 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20908, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:38.856324+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:38.856324+0000', 'last_peered': '2024-09-11T01:27:38.856324+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:38.856324+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:41.503 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:41.981 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:41.981 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8687/24 objects degraded (36195.833%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":862,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovering+undersized+degraded+remapped","count":2},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1463709696,"bytes_avail":288446582784,"bytes_total":289910292480,"degraded_objects":8687,"degraded_total":24,"degraded_ratio":361.95833333333326,"recovering_objects_per_sec":155,"recovering_bytes_per_sec":120,"recovering_keys_per_sec":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:41.996 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:41.996 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:42.353 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:42.353 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:42.366 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-11T01:27:40.688031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40948,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1429404,"kb_used_data":116832,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281686116,"statfs":{"total":289910292480,"available":288446582784,"internally_reserved":0,"allocated":119635968,"data_stored":118576299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1244,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1244,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001442"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28600,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856058+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856058+0000","last_peered":"2024-09-11T01:27:38.856058+0000","last_clean":"2024-09-11T01:27:38.856058+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856058+0000","last_undegraded":"2024-09-11T01:27:38.856058+0000","last_fullsized":"2024-09-11T01:27:38.856058+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20633,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761385+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:11.761385+0000","last_peered":"2024-09-11T01:27:11.761385+0000","last_clean":"2024-09-11T01:27:11.761385+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:11.761385+0000","last_undegraded":"2024-09-11T01:27:11.761385+0000","last_fullsized":"2024-09-11T01:27:11.761385+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20241,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762189+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:11.762189+0000","last_peered":"2024-09-11T01:27:11.762189+0000","last_clean":"2024-09-11T01:27:11.762189+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:11.762189+0000","last_undegraded":"2024-09-11T01:27:11.762189+0000","last_fullsized":"2024-09-11T01:27:11.762189+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20438,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761575+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:11.761575+0000","last_peered":"2024-09-11T01:27:11.761575+0000","last_clean":"2024-09-11T01:27:11.761575+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:11.761575+0000","last_undegraded":"2024-09-11T01:27:11.761575+0000","last_fullsized":"2024-09-11T01:27:11.761575+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":27094,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:34.094491+0000","last_change":"2024-09-11T01:27:10.304340+0000","last_active":"2024-09-11T01:27:34.094491+0000","last_peered":"2024-09-11T01:27:34.094491+0000","last_clean":"2024-09-11T01:15:17.953735+0000","last_became_active":"2024-09-11T01:15:20.541862+0000","last_became_peered":"2024-09-11T01:15:20.541862+0000","last_unstale":"2024-09-11T01:27:34.094491+0000","last_undegraded":"2024-09-11T01:15:20.084964+0000","last_fullsized":"2024-09-11T01:15:19.985420+0000","mapping_epoch":38,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":756,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3360,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-755}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21750,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:39.093437+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:39.093437+0000","last_peered":"2024-09-11T01:27:39.093437+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:39.093437+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3897}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762159+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:11.762159+0000","last_peered":"2024-09-11T01:27:11.762159+0000","last_clean":"2024-09-11T01:27:11.762159+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:11.762159+0000","last_undegraded":"2024-09-11T01:27:11.762159+0000","last_fullsized":"2024-09-11T01:27:11.762159+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28623,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762569+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:11.762569+0000","last_peered":"2024-09-11T01:27:11.762569+0000","last_clean":"2024-09-11T01:27:11.762569+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:11.762569+0000","last_undegraded":"2024-09-11T01:27:11.762569+0000","last_fullsized":"2024-09-11T01:27:11.762569+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761627+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:11.761627+0000","last_peered":"2024-09-11T01:27:11.761627+0000","last_clean":"2024-09-11T01:27:11.761627+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:11.761627+0000","last_undegraded":"2024-09-11T01:27:11.761627+0000","last_fullsized":"2024-09-11T01:27:11.761627+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20908,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.856324+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.856324+0000","last_peered":"2024-09-11T01:27:38.856324+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.856324+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28328,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761550+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:11.761550+0000","last_peered":"2024-09-11T01:27:11.761550+0000","last_clean":"2024-09-11T01:27:11.761550+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:11.761550+0000","last_undegraded":"2024-09-11T01:27:11.761550+0000","last_fullsized":"2024-09-11T01:27:11.761550+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762477+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:11.762477+0000","last_peered":"2024-09-11T01:27:11.762477+0000","last_clean":"2024-09-11T01:27:11.762477+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:11.762477+0000","last_undegraded":"2024-09-11T01:27:11.762477+0000","last_fullsized":"2024-09-11T01:27:11.762477+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29053,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761251+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:11.761251+0000","last_peered":"2024-09-11T01:27:11.761251+0000","last_clean":"2024-09-11T01:27:11.761251+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:11.761251+0000","last_undegraded":"2024-09-11T01:27:11.761251+0000","last_fullsized":"2024-09-11T01:27:11.761251+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28854,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762458+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:11.762458+0000","last_peered":"2024-09-11T01:27:11.762458+0000","last_clean":"2024-09-11T01:27:11.762458+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:11.762458+0000","last_undegraded":"2024-09-11T01:27:11.762458+0000","last_fullsized":"2024-09-11T01:27:11.762458+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762670+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:11.762670+0000","last_peered":"2024-09-11T01:27:11.762670+0000","last_clean":"2024-09-11T01:27:11.762670+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:11.762670+0000","last_undegraded":"2024-09-11T01:27:11.762670+0000","last_fullsized":"2024-09-11T01:27:11.762670+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20949,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.865565+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:11.865565+0000","last_peered":"2024-09-11T01:27:11.865565+0000","last_clean":"2024-09-11T01:27:11.865565+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:11.865565+0000","last_undegraded":"2024-09-11T01:27:11.865565+0000","last_fullsized":"2024-09-11T01:27:11.865565+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":92,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.761686+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:11.761686+0000","last_peered":"2024-09-11T01:27:11.761686+0000","last_clean":"2024-09-11T01:27:11.761686+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:11.761686+0000","last_undegraded":"2024-09-11T01:27:11.761686+0000","last_fullsized":"2024-09-11T01:27:11.761686+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20922,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762293+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:11.762293+0000","last_peered":"2024-09-11T01:27:11.762293+0000","last_clean":"2024-09-11T01:27:11.762293+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:11.762293+0000","last_undegraded":"2024-09-11T01:27:11.762293+0000","last_fullsized":"2024-09-11T01:27:11.762293+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762468+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:11.762468+0000","last_peered":"2024-09-11T01:27:11.762468+0000","last_clean":"2024-09-11T01:27:11.762468+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:11.762468+0000","last_undegraded":"2024-09-11T01:27:11.762468+0000","last_fullsized":"2024-09-11T01:27:11.762468+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762306+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:11.762306+0000","last_peered":"2024-09-11T01:27:11.762306+0000","last_clean":"2024-09-11T01:27:11.762306+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:11.762306+0000","last_undegraded":"2024-09-11T01:27:11.762306+0000","last_fullsized":"2024-09-11T01:27:11.762306+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20406,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:27:11.762600+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:11.762600+0000","last_peered":"2024-09-11T01:27:11.762600+0000","last_clean":"2024-09-11T01:27:11.762600+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:11.762600+0000","last_undegraded":"2024-09-11T01:27:11.762600+0000","last_fullsized":"2024-09-11T01:27:11.762600+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8687,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40947,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118194176,"data_stored":118096625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393172,"kb_used_data":50492,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93978668,"statfs":{"total":96636764160,"available":96234156032,"internally_reserved":0,"allocated":51703808,"data_stored":51358721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790100,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527816,"kb_used_data":504,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844024,"statfs":{"total":96636764160,"available":96096280576,"internally_reserved":0,"allocated":516096,"data_stored":163319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508416,"kb_used_data":65836,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863424,"statfs":{"total":96636764160,"available":96116146176,"internally_reserved":0,"allocated":67416064,"data_stored":67054259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":51228672,"data_stored":51199768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:45.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:45.771 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:45.771 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:45.785 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T01:27:44.688861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42752,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1400508,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281715012,"statfs":{"total":289910292480,"available":288476172288,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001562"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":23846,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:44.095216+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:44.095216+0000","last_peered":"2024-09-11T01:27:44.095216+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:44.095216+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2851,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1432,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855355+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855355+0000","last_peered":"2024-09-11T01:27:38.855355+0000","last_clean":"2024-09-11T01:27:38.855355+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855355+0000","last_undegraded":"2024-09-11T01:27:38.855355+0000","last_fullsized":"2024-09-11T01:27:38.855355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856112+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856112+0000","last_peered":"2024-09-11T01:27:38.856112+0000","last_clean":"2024-09-11T01:27:38.856112+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856112+0000","last_undegraded":"2024-09-11T01:27:38.856112+0000","last_fullsized":"2024-09-11T01:27:38.856112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42751,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856488,"statfs":{"total":96636764160,"available":96109043712,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:45.787 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:46.211 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:46.212 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:46.225 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T01:27:44.688861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42752,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1400508,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281715012,"statfs":{"total":289910292480,"available":288476172288,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001562"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":23846,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:44.095216+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:44.095216+0000","last_peered":"2024-09-11T01:27:44.095216+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:44.095216+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2851,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1432,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855355+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855355+0000","last_peered":"2024-09-11T01:27:38.855355+0000","last_clean":"2024-09-11T01:27:38.855355+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855355+0000","last_undegraded":"2024-09-11T01:27:38.855355+0000","last_fullsized":"2024-09-11T01:27:38.855355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856112+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856112+0000","last_peered":"2024-09-11T01:27:38.856112+0000","last_clean":"2024-09-11T01:27:38.856112+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856112+0000","last_undegraded":"2024-09-11T01:27:38.856112+0000","last_fullsized":"2024-09-11T01:27:38.856112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42751,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856488,"statfs":{"total":96636764160,"available":96109043712,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:46.226 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:46.226 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 23846, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:44.095216+0000', 'last_change': '2024-09-11T01:27:36.975917+0000', 'last_active': '2024-09-11T01:27:44.095216+0000', 'last_peered': '2024-09-11T01:27:44.095216+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:44.095216+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2851, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1432, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2849}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:46.226 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:46.227 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20909, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:38.857352+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:38.857352+0000', 'last_peered': '2024-09-11T01:27:38.857352+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:38.857352+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:46.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:46.718 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:46.718 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6883/24 objects degraded (28679.167%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":867,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1434120192,"bytes_avail":288476172288,"bytes_total":289910292480,"degraded_objects":6883,"degraded_total":24,"degraded_ratio":286.79166666666669,"recovering_objects_per_sec":263,"recovering_bytes_per_sec":120,"recovering_keys_per_sec":0,"num_objects_recovered":2111,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:46.736 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:46.736 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:47.085 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:47.086 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:47.097 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-11T01:27:46.689255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42752,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1400508,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281715012,"statfs":{"total":289910292480,"available":288476172288,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1331219463},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.841582"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":23846,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:44.095216+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:44.095216+0000","last_peered":"2024-09-11T01:27:44.095216+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:44.095216+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2851,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1432,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855355+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855355+0000","last_peered":"2024-09-11T01:27:38.855355+0000","last_clean":"2024-09-11T01:27:38.855355+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855355+0000","last_undegraded":"2024-09-11T01:27:38.855355+0000","last_fullsized":"2024-09-11T01:27:38.855355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856112+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856112+0000","last_peered":"2024-09-11T01:27:38.856112+0000","last_clean":"2024-09-11T01:27:38.856112+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856112+0000","last_undegraded":"2024-09-11T01:27:38.856112+0000","last_fullsized":"2024-09-11T01:27:38.856112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42751,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790101,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856488,"statfs":{"total":96636764160,"available":96109043712,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:50.100 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:50.483 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:50.484 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:50.497 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T01:27:48.689604+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42752,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1408828,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281706692,"statfs":{"total":289910292480,"available":288467652608,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1339739143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2111,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2111,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001936"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":23846,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:44.095216+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:44.095216+0000","last_peered":"2024-09-11T01:27:44.095216+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:44.095216+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2851,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1432,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2849}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855355+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855355+0000","last_peered":"2024-09-11T01:27:38.855355+0000","last_clean":"2024-09-11T01:27:38.855355+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855355+0000","last_undegraded":"2024-09-11T01:27:38.855355+0000","last_fullsized":"2024-09-11T01:27:38.855355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856112+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856112+0000","last_peered":"2024-09-11T01:27:38.856112+0000","last_clean":"2024-09-11T01:27:38.856112+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856112+0000","last_undegraded":"2024-09-11T01:27:38.856112+0000","last_fullsized":"2024-09-11T01:27:38.856112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42751,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790102,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523672,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848168,"statfs":{"total":96636764160,"available":96100524032,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:50.498 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:50.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:50.833 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:50.849 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-11T01:27:50.689919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43938,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1408828,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281706692,"statfs":{"total":289910292480,"available":288467652608,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1339739143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3297,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.837910"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":26218,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:49.099740+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:49.099740+0000","last_peered":"2024-09-11T01:27:49.099740+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:49.099740+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2618,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855355+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855355+0000","last_peered":"2024-09-11T01:27:38.855355+0000","last_clean":"2024-09-11T01:27:38.855355+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855355+0000","last_undegraded":"2024-09-11T01:27:38.855355+0000","last_fullsized":"2024-09-11T01:27:38.855355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856112+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856112+0000","last_peered":"2024-09-11T01:27:38.856112+0000","last_clean":"2024-09-11T01:27:38.856112+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856112+0000","last_undegraded":"2024-09-11T01:27:38.856112+0000","last_fullsized":"2024-09-11T01:27:38.856112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43937,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790102,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523672,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848168,"statfs":{"total":96636764160,"available":96100524032,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:50.851 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:50.851 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 26218, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:49.099740+0000', 'last_change': '2024-09-11T01:27:36.975917+0000', 'last_active': '2024-09-11T01:27:49.099740+0000', 'last_peered': '2024-09-11T01:27:49.099740+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:49.099740+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1665, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2618, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:50.851 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:50.851 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20909, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:38.857352+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:38.857352+0000', 'last_peered': '2024-09-11T01:27:38.857352+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:38.857352+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:50.851 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:51.258 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:51.258 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5697/24 objects degraded (23737.500%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":871,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1442639872,"bytes_avail":288467652608,"bytes_total":289910292480,"degraded_objects":5697,"degraded_total":24,"degraded_ratio":237.375,"recovering_objects_per_sec":278,"recovering_bytes_per_sec":81,"recovering_keys_per_sec":0,"num_objects_recovered":3297,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:51.271 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:51.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:51.679 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:51.680 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:51.693 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":488,"stamp":"2024-09-11T01:27:50.689919+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43938,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1408828,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281706692,"statfs":{"total":289910292480,"available":288467652608,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1339739143},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3297,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.837910"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":26218,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:49.099740+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:49.099740+0000","last_peered":"2024-09-11T01:27:49.099740+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:49.099740+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2618,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855355+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855355+0000","last_peered":"2024-09-11T01:27:38.855355+0000","last_clean":"2024-09-11T01:27:38.855355+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855355+0000","last_undegraded":"2024-09-11T01:27:38.855355+0000","last_fullsized":"2024-09-11T01:27:38.855355+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856112+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856112+0000","last_peered":"2024-09-11T01:27:38.856112+0000","last_clean":"2024-09-11T01:27:38.856112+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856112+0000","last_undegraded":"2024-09-11T01:27:38.856112+0000","last_fullsized":"2024-09-11T01:27:38.856112+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5697,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43937,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790102,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523672,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848168,"statfs":{"total":96636764160,"available":96100524032,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:54.696 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:55.055 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:55.056 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:55.069 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-11T01:27:54.920596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45424,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1404668,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281710852,"statfs":{"total":289910292480,"available":288471912448,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.231735"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":29190,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:54.101460+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:54.101460+0000","last_peered":"2024-09-11T01:27:54.101460+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:54.101460+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4104,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855404+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855404+0000","last_peered":"2024-09-11T01:27:38.855404+0000","last_clean":"2024-09-11T01:27:38.855404+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855404+0000","last_undegraded":"2024-09-11T01:27:38.855404+0000","last_fullsized":"2024-09-11T01:27:38.855404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856149+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856149+0000","last_peered":"2024-09-11T01:27:38.856149+0000","last_clean":"2024-09-11T01:27:38.856149+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856149+0000","last_undegraded":"2024-09-11T01:27:38.856149+0000","last_fullsized":"2024-09-11T01:27:38.856149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45423,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790103,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852328,"statfs":{"total":96636764160,"available":96104783872,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:55.071 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:55.404 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:55.417 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-11T01:27:54.920596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45424,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1404668,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281710852,"statfs":{"total":289910292480,"available":288471912448,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.231735"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":29190,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:54.101460+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:54.101460+0000","last_peered":"2024-09-11T01:27:54.101460+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:54.101460+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4104,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855404+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855404+0000","last_peered":"2024-09-11T01:27:38.855404+0000","last_clean":"2024-09-11T01:27:38.855404+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855404+0000","last_undegraded":"2024-09-11T01:27:38.855404+0000","last_fullsized":"2024-09-11T01:27:38.855404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856149+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856149+0000","last_peered":"2024-09-11T01:27:38.856149+0000","last_clean":"2024-09-11T01:27:38.856149+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856149+0000","last_undegraded":"2024-09-11T01:27:38.856149+0000","last_fullsized":"2024-09-11T01:27:38.856149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45423,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790103,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852328,"statfs":{"total":96636764160,"available":96104783872,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:55.418 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:27:55.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'20812", 'reported_seq': 29190, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:54.101460+0000', 'last_change': '2024-09-11T01:27:36.975917+0000', 'last_active': '2024-09-11T01:27:54.101460+0000', 'last_peered': '2024-09-11T01:27:54.101460+0000', 'last_clean': '2024-09-11T01:15:17.937255+0000', 'last_became_active': '2024-09-11T01:15:20.540780+0000', 'last_became_peered': '2024-09-11T01:15:20.540780+0000', 'last_unstale': '2024-09-11T01:27:54.101460+0000', 'last_undegraded': '2024-09-11T01:15:20.098932+0000', 'last_fullsized': '2024-09-11T01:15:19.985129+0000', 'mapping_epoch': 38, 'log_start': "41'10800", 'ondisk_log_start': "41'10800", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3306", 'last_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:03.099483+0000', 'objects_scrubbed': 3243, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1279, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 179, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 2, 'num_read_kb': 2, 'num_write': 17565, 'num_write_kb': 28648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4104, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -177}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:55.419 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:55.419 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20909, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:38.857352+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:38.857352+0000', 'last_peered': '2024-09-11T01:27:38.857352+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:38.857352+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:55.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:55.861 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:55.861 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4211/24 objects degraded (17545.833%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":876,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1438380032,"bytes_avail":288471912448,"bytes_total":289910292480,"degraded_objects":4211,"degraded_total":24,"degraded_ratio":175.45833333333337,"recovering_objects_per_sec":309,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":3720,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:55.875 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:55.875 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:56.303 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:56.304 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:56.319 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-11T01:27:55.918595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45424,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1404668,"kb_used_data":100416,"kb_used_omap":12,"kb_used_meta":1304179,"kb_avail":281710852,"statfs":{"total":289910292480,"available":288471912448,"internally_reserved":0,"allocated":102825984,"data_stored":101742790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1335479303},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2672,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2672,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.229340"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20637,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.854051+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:38.854051+0000","last_peered":"2024-09-11T01:27:38.854051+0000","last_clean":"2024-09-11T01:27:38.854051+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:38.854051+0000","last_undegraded":"2024-09-11T01:27:38.854051+0000","last_fullsized":"2024-09-11T01:27:38.854051+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20245,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855306+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:38.855306+0000","last_peered":"2024-09-11T01:27:38.855306+0000","last_clean":"2024-09-11T01:27:38.855306+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:38.855306+0000","last_undegraded":"2024-09-11T01:27:38.855306+0000","last_fullsized":"2024-09-11T01:27:38.855306+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20442,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853812+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:38.853812+0000","last_peered":"2024-09-11T01:27:38.853812+0000","last_clean":"2024-09-11T01:27:38.853812+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:38.853812+0000","last_undegraded":"2024-09-11T01:27:38.853812+0000","last_fullsized":"2024-09-11T01:27:38.853812+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20486,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.965773+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:38.965773+0000","last_peered":"2024-09-11T01:27:38.965773+0000","last_clean":"2024-09-11T01:27:38.965773+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:38.965773+0000","last_undegraded":"2024-09-11T01:27:38.965773+0000","last_fullsized":"2024-09-11T01:27:38.965773+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":29190,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:54.101460+0000","last_change":"2024-09-11T01:27:36.975917+0000","last_active":"2024-09-11T01:27:54.101460+0000","last_peered":"2024-09-11T01:27:54.101460+0000","last_clean":"2024-09-11T01:15:17.937255+0000","last_became_active":"2024-09-11T01:15:20.540780+0000","last_became_peered":"2024-09-11T01:15:20.540780+0000","last_unstale":"2024-09-11T01:27:54.101460+0000","last_undegraded":"2024-09-11T01:15:20.098932+0000","last_fullsized":"2024-09-11T01:15:19.985129+0000","mapping_epoch":38,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":179,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4104,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855167+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:38.855167+0000","last_peered":"2024-09-11T01:27:38.855167+0000","last_clean":"2024-09-11T01:27:38.855167+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:38.855167+0000","last_undegraded":"2024-09-11T01:27:38.855167+0000","last_fullsized":"2024-09-11T01:27:38.855167+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28627,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855468+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:38.855468+0000","last_peered":"2024-09-11T01:27:38.855468+0000","last_clean":"2024-09-11T01:27:38.855468+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:38.855468+0000","last_undegraded":"2024-09-11T01:27:38.855468+0000","last_fullsized":"2024-09-11T01:27:38.855468+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853487+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:38.853487+0000","last_peered":"2024-09-11T01:27:38.853487+0000","last_clean":"2024-09-11T01:27:38.853487+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:38.853487+0000","last_undegraded":"2024-09-11T01:27:38.853487+0000","last_fullsized":"2024-09-11T01:27:38.853487+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28332,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853562+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:38.853562+0000","last_peered":"2024-09-11T01:27:38.853562+0000","last_clean":"2024-09-11T01:27:38.853562+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:38.853562+0000","last_undegraded":"2024-09-11T01:27:38.853562+0000","last_fullsized":"2024-09-11T01:27:38.853562+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855404+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:38.855404+0000","last_peered":"2024-09-11T01:27:38.855404+0000","last_clean":"2024-09-11T01:27:38.855404+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:38.855404+0000","last_undegraded":"2024-09-11T01:27:38.855404+0000","last_fullsized":"2024-09-11T01:27:38.855404+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29057,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853252+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:38.853252+0000","last_peered":"2024-09-11T01:27:38.853252+0000","last_clean":"2024-09-11T01:27:38.853252+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:38.853252+0000","last_undegraded":"2024-09-11T01:27:38.853252+0000","last_fullsized":"2024-09-11T01:27:38.853252+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28858,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855745+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:38.855745+0000","last_peered":"2024-09-11T01:27:38.855745+0000","last_clean":"2024-09-11T01:27:38.855745+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:38.855745+0000","last_undegraded":"2024-09-11T01:27:38.855745+0000","last_fullsized":"2024-09-11T01:27:38.855745+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856149+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:38.856149+0000","last_peered":"2024-09-11T01:27:38.856149+0000","last_clean":"2024-09-11T01:27:38.856149+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:38.856149+0000","last_undegraded":"2024-09-11T01:27:38.856149+0000","last_fullsized":"2024-09-11T01:27:38.856149+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20953,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855578+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:38.855578+0000","last_peered":"2024-09-11T01:27:38.855578+0000","last_clean":"2024-09-11T01:27:38.855578+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:38.855578+0000","last_undegraded":"2024-09-11T01:27:38.855578+0000","last_fullsized":"2024-09-11T01:27:38.855578+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.853961+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:38.853961+0000","last_peered":"2024-09-11T01:27:38.853961+0000","last_clean":"2024-09-11T01:27:38.853961+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:38.853961+0000","last_undegraded":"2024-09-11T01:27:38.853961+0000","last_fullsized":"2024-09-11T01:27:38.853961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20926,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855398+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:38.855398+0000","last_peered":"2024-09-11T01:27:38.855398+0000","last_clean":"2024-09-11T01:27:38.855398+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:38.855398+0000","last_undegraded":"2024-09-11T01:27:38.855398+0000","last_fullsized":"2024-09-11T01:27:38.855398+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855503+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:38.855503+0000","last_peered":"2024-09-11T01:27:38.855503+0000","last_clean":"2024-09-11T01:27:38.855503+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:38.855503+0000","last_undegraded":"2024-09-11T01:27:38.855503+0000","last_fullsized":"2024-09-11T01:27:38.855503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855672+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:38.855672+0000","last_peered":"2024-09-11T01:27:38.855672+0000","last_clean":"2024-09-11T01:27:38.855672+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:38.855672+0000","last_undegraded":"2024-09-11T01:27:38.855672+0000","last_fullsized":"2024-09-11T01:27:38.855672+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20410,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856016+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:38.856016+0000","last_peered":"2024-09-11T01:27:38.856016+0000","last_clean":"2024-09-11T01:27:38.856016+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:38.856016+0000","last_undegraded":"2024-09-11T01:27:38.856016+0000","last_fullsized":"2024-09-11T01:27:38.856016+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4211,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45423,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":101343232,"data_stored":101249294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":376732,"kb_used_data":34052,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93995108,"statfs":{"total":96636764160,"available":96250990592,"internally_reserved":0,"allocated":34869248,"data_stored":34516923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790103,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852328,"statfs":{"total":96636764160,"available":96104783872,"internally_reserved":0,"allocated":532480,"data_stored":168852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34377728,"data_stored":34352437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:59.322 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:59.686 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:27:59.686 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:27:59.700 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-11T01:27:58.691239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45603,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415020,"kb_used_data":98164,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281700500,"statfs":{"total":289910292480,"available":288461312000,"internally_reserved":0,"allocated":100519936,"data_stored":99422166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1348258823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001319"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920466+0000","last_peered":"2024-09-11T01:27:55.920466+0000","last_clean":"2024-09-11T01:27:55.920466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920466+0000","last_undegraded":"2024-09-11T01:27:55.920466+0000","last_fullsized":"2024-09-11T01:27:55.920466+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920901+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.920901+0000","last_peered":"2024-09-11T01:27:55.920901+0000","last_clean":"2024-09-11T01:27:55.920901+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.920901+0000","last_undegraded":"2024-09-11T01:27:55.920901+0000","last_fullsized":"2024-09-11T01:27:55.920901+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20414,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920569+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920569+0000","last_peered":"2024-09-11T01:27:55.920569+0000","last_clean":"2024-09-11T01:27:55.920569+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920569+0000","last_undegraded":"2024-09-11T01:27:55.920569+0000","last_fullsized":"2024-09-11T01:27:55.920569+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45602,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374588,"kb_used_data":31792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93997252,"statfs":{"total":96636764160,"available":96253186048,"internally_reserved":0,"allocated":32555008,"data_stored":32191782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790104,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532008,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":531451,"kb_avail":93839832,"statfs":{"total":96636764160,"available":96091987968,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":544206511},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:59.701 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:00.124 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:00.125 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:00.137 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-11T01:27:58.691239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45603,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415020,"kb_used_data":98164,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281700500,"statfs":{"total":289910292480,"available":288461312000,"internally_reserved":0,"allocated":100519936,"data_stored":99422166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1348258823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1665,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001319"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28601,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856099+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:38.856099+0000","last_peered":"2024-09-11T01:27:38.856099+0000","last_clean":"2024-09-11T01:27:38.856099+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:38.856099+0000","last_undegraded":"2024-09-11T01:27:38.856099+0000","last_fullsized":"2024-09-11T01:27:38.856099+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20434,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855687+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:38.855687+0000","last_peered":"2024-09-11T01:27:38.855687+0000","last_clean":"2024-09-11T01:27:38.855687+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:38.855687+0000","last_undegraded":"2024-09-11T01:27:38.855687+0000","last_fullsized":"2024-09-11T01:27:38.855687+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20579,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.855589+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:38.855589+0000","last_peered":"2024-09-11T01:27:38.855589+0000","last_clean":"2024-09-11T01:27:38.855589+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:38.855589+0000","last_undegraded":"2024-09-11T01:27:38.855589+0000","last_fullsized":"2024-09-11T01:27:38.855589+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920466+0000","last_peered":"2024-09-11T01:27:55.920466+0000","last_clean":"2024-09-11T01:27:55.920466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920466+0000","last_undegraded":"2024-09-11T01:27:55.920466+0000","last_fullsized":"2024-09-11T01:27:55.920466+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":20909,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:38.857352+0000","last_change":"2024-09-11T01:15:20.556267+0000","last_active":"2024-09-11T01:27:38.857352+0000","last_peered":"2024-09-11T01:27:38.857352+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:38.857352+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":75,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4030}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:27:38.856035+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:38.856035+0000","last_peered":"2024-09-11T01:27:38.856035+0000","last_clean":"2024-09-11T01:27:38.856035+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:38.856035+0000","last_undegraded":"2024-09-11T01:27:38.856035+0000","last_fullsized":"2024-09-11T01:27:38.856035+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920901+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.920901+0000","last_peered":"2024-09-11T01:27:55.920901+0000","last_clean":"2024-09-11T01:27:55.920901+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.920901+0000","last_undegraded":"2024-09-11T01:27:55.920901+0000","last_fullsized":"2024-09-11T01:27:55.920901+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20414,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920569+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920569+0000","last_peered":"2024-09-11T01:27:55.920569+0000","last_clean":"2024-09-11T01:27:55.920569+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920569+0000","last_undegraded":"2024-09-11T01:27:55.920569+0000","last_fullsized":"2024-09-11T01:27:55.920569+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45602,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374588,"kb_used_data":31792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93997252,"statfs":{"total":96636764160,"available":96253186048,"internally_reserved":0,"allocated":32555008,"data_stored":32191782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790104,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532008,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":531451,"kb_avail":93839832,"statfs":{"total":96636764160,"available":96091987968,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":544206511},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":65844,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":67424256,"data_stored":67057015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:00.138 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:28:00.138 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 20909, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:38.857352+0000', 'last_change': '2024-09-11T01:15:20.556267+0000', 'last_active': '2024-09-11T01:27:38.857352+0000', 'last_peered': '2024-09-11T01:27:38.857352+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:27:38.857352+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4032, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 75, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4030}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:28:00.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:28:00.613 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:00.614 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4032/24 objects degraded (16800.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":881,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1448980480,"bytes_avail":288461312000,"bytes_total":289910292480,"degraded_objects":4032,"degraded_total":24,"degraded_ratio":168,"recovering_objects_per_sec":208,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1665,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:28:00.629 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:28:00.629 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:00.970 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:00.970 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:00.986 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-11T01:28:00.691548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46409,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1415040,"kb_used_data":98180,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281700480,"statfs":{"total":289910292480,"available":288461291520,"internally_reserved":0,"allocated":100536320,"data_stored":99427649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1348258823},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2471,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2471,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001319"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920466+0000","last_peered":"2024-09-11T01:27:55.920466+0000","last_clean":"2024-09-11T01:27:55.920466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920466+0000","last_undegraded":"2024-09-11T01:27:55.920466+0000","last_fullsized":"2024-09-11T01:27:55.920466+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":22526,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:59.098639+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:27:59.098639+0000","last_peered":"2024-09-11T01:27:59.098639+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:59.098639+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":881,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920901+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.920901+0000","last_peered":"2024-09-11T01:27:55.920901+0000","last_clean":"2024-09-11T01:27:55.920901+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.920901+0000","last_undegraded":"2024-09-11T01:27:55.920901+0000","last_fullsized":"2024-09-11T01:27:55.920901+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20414,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920569+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920569+0000","last_peered":"2024-09-11T01:27:55.920569+0000","last_clean":"2024-09-11T01:27:55.920569+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920569+0000","last_undegraded":"2024-09-11T01:27:55.920569+0000","last_fullsized":"2024-09-11T01:27:55.920569+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46408,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374588,"kb_used_data":31792,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93997252,"statfs":{"total":96636764160,"available":96253186048,"internally_reserved":0,"allocated":32555008,"data_stored":32191782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":350875310},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790104,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532008,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":531451,"kb_avail":93839832,"statfs":{"total":96636764160,"available":96091987968,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":544206511},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:03.989 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:04.356 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:04.357 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:04.370 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-11T01:28:02.691840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46409,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395896,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281719624,"statfs":{"total":289910292480,"available":288480894976,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":985,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001234"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920466+0000","last_peered":"2024-09-11T01:27:55.920466+0000","last_clean":"2024-09-11T01:27:55.920466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920466+0000","last_undegraded":"2024-09-11T01:27:55.920466+0000","last_fullsized":"2024-09-11T01:27:55.920466+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":22526,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:59.098639+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:27:59.098639+0000","last_peered":"2024-09-11T01:27:59.098639+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:27:59.098639+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":881,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3226,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46408,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790105,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:04.371 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:04.715 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:04.715 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:04.728 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-11T01:28:04.692160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47502,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395896,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281719624,"statfs":{"total":289910292480,"available":288480894976,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.771563"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920466+0000","last_peered":"2024-09-11T01:27:55.920466+0000","last_clean":"2024-09-11T01:27:55.920466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920466+0000","last_undegraded":"2024-09-11T01:27:55.920466+0000","last_fullsized":"2024-09-11T01:27:55.920466+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":24712,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:04.106118+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:04.106118+0000","last_peered":"2024-09-11T01:28:04.106118+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:04.106118+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47501,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790105,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:04.729 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:28:04.729 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 24712, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:28:04.106118+0000', 'last_change': '2024-09-11T01:27:54.654949+0000', 'last_active': '2024-09-11T01:28:04.106118+0000', 'last_peered': '2024-09-11T01:28:04.106118+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:28:04.106118+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1974, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2133}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:28:04.729 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:28:05.208 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:05.209 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3226/24 objects degraded (13441.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":885,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1429397504,"bytes_avail":288480894976,"bytes_total":289910292480,"degraded_objects":3226,"degraded_total":24,"degraded_ratio":134.41666666666666,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":985,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:28:05.224 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:28:05.225 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:05.567 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:05.567 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:05.580 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-11T01:28:04.692160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47502,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1395896,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281719624,"statfs":{"total":289910292480,"available":288480894976,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1343998983},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.771563"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28631,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920466+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920466+0000","last_peered":"2024-09-11T01:27:55.920466+0000","last_clean":"2024-09-11T01:27:55.920466+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920466+0000","last_undegraded":"2024-09-11T01:27:55.920466+0000","last_fullsized":"2024-09-11T01:27:55.920466+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":24712,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:04.106118+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:04.106118+0000","last_peered":"2024-09-11T01:28:04.106118+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:04.106118+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47501,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790105,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523680,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848160,"statfs":{"total":96636764160,"available":96100515840,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:08.583 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:08.899 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:08.900 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:08.912 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-11T01:28:08.692777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47502,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1405624,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281709896,"statfs":{"total":289910292480,"available":288470933504,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1353960455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001845"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":24712,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:04.106118+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:04.106118+0000","last_peered":"2024-09-11T01:28:04.106118+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:04.106118+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47501,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838432,"statfs":{"total":96636764160,"available":96090554368,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:08.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:09.242 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:09.243 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:09.256 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-11T01:28:08.692777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47502,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1405624,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281709896,"statfs":{"total":289910292480,"available":288470933504,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1353960455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001845"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":24712,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:04.106118+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:04.106118+0000","last_peered":"2024-09-11T01:28:04.106118+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:04.106118+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47501,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838432,"statfs":{"total":96636764160,"available":96090554368,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:09.257 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:28:09.257 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 24712, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:28:04.106118+0000', 'last_change': '2024-09-11T01:27:54.654949+0000', 'last_active': '2024-09-11T01:28:04.106118+0000', 'last_peered': '2024-09-11T01:28:04.106118+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:28:04.106118+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2133, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1974, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2133}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:28:09.257 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:28:09.646 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:09.647 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2133/24 objects degraded (8887.500%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":890,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1439358976,"bytes_avail":288470933504,"bytes_total":289910292480,"degraded_objects":2133,"degraded_total":24,"degraded_ratio":88.875,"recovering_objects_per_sec":173,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2078,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:28:09.660 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:28:09.660 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:09.994 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:09.994 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:10.007 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-11T01:28:08.692777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47502,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1405624,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1322227,"kb_avail":281709896,"statfs":{"total":289910292480,"available":288470933504,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1353960455},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2078,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001845"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":24712,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:04.106118+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:04.106118+0000","last_peered":"2024-09-11T01:28:04.106118+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:04.106118+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1974,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2133,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47501,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790106,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838432,"statfs":{"total":96636764160,"available":96090554368,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:13.010 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:13.321 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:13.322 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:13.334 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-11T01:28:12.693448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48489,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413944,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281701576,"statfs":{"total":289910292480,"available":288462413824,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1362480135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2080,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001899"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":26686,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:09.101344+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:09.101344+0000","last_peered":"2024-09-11T01:28:09.101344+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:09.101344+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2961,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48488,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790107,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830112,"statfs":{"total":96636764160,"available":96082034688,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:13.337 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:13.648 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:13.649 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:13.661 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-11T01:28:12.693448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48489,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413944,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281701576,"statfs":{"total":289910292480,"available":288462413824,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1362480135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2080,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001899"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":26686,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:09.101344+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:09.101344+0000","last_peered":"2024-09-11T01:28:09.101344+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:09.101344+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2961,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48488,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790107,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830112,"statfs":{"total":96636764160,"available":96082034688,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:13.662 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:28:13.662 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 26686, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:28:09.101344+0000', 'last_change': '2024-09-11T01:27:54.654949+0000', 'last_active': '2024-09-11T01:28:09.101344+0000', 'last_peered': '2024-09-11T01:28:09.101344+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:28:09.101344+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1146, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2961, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1146}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:28:13.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:28:14.145 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:14.146 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1146/24 objects degraded (4775.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":894,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1447878656,"bytes_avail":288462413824,"bytes_total":289910292480,"degraded_objects":1146,"degraded_total":24,"degraded_ratio":47.75,"recovering_objects_per_sec":173,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2080,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:28:14.159 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:28:14.159 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:14.460 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:14.472 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-11T01:28:12.693448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48489,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1413944,"kb_used_data":83324,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281701576,"statfs":{"total":289910292480,"available":288462413824,"internally_reserved":0,"allocated":85323776,"data_stored":84222331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1362480135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2080,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001899"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20642,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921122+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:27:55.921122+0000","last_peered":"2024-09-11T01:27:55.921122+0000","last_clean":"2024-09-11T01:27:55.921122+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:27:55.921122+0000","last_undegraded":"2024-09-11T01:27:55.921122+0000","last_fullsized":"2024-09-11T01:27:55.921122+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20447,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921016+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:27:55.921016+0000","last_peered":"2024-09-11T01:27:55.921016+0000","last_clean":"2024-09-11T01:27:55.921016+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:27:55.921016+0000","last_undegraded":"2024-09-11T01:27:55.921016+0000","last_fullsized":"2024-09-11T01:27:55.921016+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921375+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:27:55.921375+0000","last_peered":"2024-09-11T01:27:55.921375+0000","last_clean":"2024-09-11T01:27:55.921375+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:27:55.921375+0000","last_undegraded":"2024-09-11T01:27:55.921375+0000","last_fullsized":"2024-09-11T01:27:55.921375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":26686,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:09.101344+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:09.101344+0000","last_peered":"2024-09-11T01:28:09.101344+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:09.101344+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2961,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28336,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921472+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:27:55.921472+0000","last_peered":"2024-09-11T01:27:55.921472+0000","last_clean":"2024-09-11T01:27:55.921472+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:27:55.921472+0000","last_undegraded":"2024-09-11T01:27:55.921472+0000","last_fullsized":"2024-09-11T01:27:55.921472+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29061,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920501+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:27:55.920501+0000","last_peered":"2024-09-11T01:27:55.920501+0000","last_clean":"2024-09-11T01:27:55.920501+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:27:55.920501+0000","last_undegraded":"2024-09-11T01:27:55.920501+0000","last_fullsized":"2024-09-11T01:27:55.920501+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":100,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920785+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:27:55.920785+0000","last_peered":"2024-09-11T01:27:55.920785+0000","last_clean":"2024-09-11T01:27:55.920785+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:27:55.920785+0000","last_undegraded":"2024-09-11T01:27:55.920785+0000","last_fullsized":"2024-09-11T01:27:55.920785+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48488,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":83791872,"data_stored":83712386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":363772,"kb_used_data":16936,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94008068,"statfs":{"total":96636764160,"available":96264261632,"internally_reserved":0,"allocated":17342464,"data_stored":16986464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790107,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830112,"statfs":{"total":96636764160,"available":96082034688,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16834560,"data_stored":16816495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:17.475 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:17.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:17.789 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:17.802 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-11T01:28:16.694344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49609,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397544,"kb_used_data":66920,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281717976,"statfs":{"total":289910292480,"available":288479207424,"internally_reserved":0,"allocated":68526080,"data_stored":67413684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1362480135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001567"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28926,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:14.104500+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:14.104500+0000","last_peered":"2024-09-11T01:28:14.104500+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:14.104500+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4081,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49608,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66977792,"data_stored":66898306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790107,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830112,"statfs":{"total":96636764160,"available":96082034688,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:17.803 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:18.099 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:18.099 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:18.111 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-11T01:28:16.694344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49609,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1397544,"kb_used_data":66920,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281717976,"statfs":{"total":289910292480,"available":288479207424,"internally_reserved":0,"allocated":68526080,"data_stored":67413684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1362480135},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001567"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20249,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920733+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:27:55.920733+0000","last_peered":"2024-09-11T01:27:55.920733+0000","last_clean":"2024-09-11T01:27:55.920733+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:27:55.920733+0000","last_undegraded":"2024-09-11T01:27:55.920733+0000","last_fullsized":"2024-09-11T01:27:55.920733+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920364+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:27:55.920364+0000","last_peered":"2024-09-11T01:27:55.920364+0000","last_clean":"2024-09-11T01:27:55.920364+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:27:55.920364+0000","last_undegraded":"2024-09-11T01:27:55.920364+0000","last_fullsized":"2024-09-11T01:27:55.920364+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:56.018524+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:27:56.018524+0000","last_peered":"2024-09-11T01:27:56.018524+0000","last_clean":"2024-09-11T01:27:56.018524+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:27:56.018524+0000","last_undegraded":"2024-09-11T01:27:56.018524+0000","last_fullsized":"2024-09-11T01:27:56.018524+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.919267+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:27:55.919267+0000","last_peered":"2024-09-11T01:27:55.919267+0000","last_clean":"2024-09-11T01:27:55.919267+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:27:55.919267+0000","last_undegraded":"2024-09-11T01:27:55.919267+0000","last_fullsized":"2024-09-11T01:27:55.919267+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28632,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920510+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:27:55.920510+0000","last_peered":"2024-09-11T01:27:55.920510+0000","last_clean":"2024-09-11T01:27:55.920510+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:27:55.920510+0000","last_undegraded":"2024-09-11T01:27:55.920510+0000","last_fullsized":"2024-09-11T01:27:55.920510+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28926,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:14.104500+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:14.104500+0000","last_peered":"2024-09-11T01:28:14.104500+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:14.104500+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4081,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920419+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:27:55.920419+0000","last_peered":"2024-09-11T01:27:55.920419+0000","last_clean":"2024-09-11T01:27:55.920419+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:27:55.920419+0000","last_undegraded":"2024-09-11T01:27:55.920419+0000","last_fullsized":"2024-09-11T01:27:55.920419+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28862,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920529+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:27:55.920529+0000","last_peered":"2024-09-11T01:27:55.920529+0000","last_clean":"2024-09-11T01:27:55.920529+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:27:55.920529+0000","last_undegraded":"2024-09-11T01:27:55.920529+0000","last_fullsized":"2024-09-11T01:27:55.920529+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920650+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:27:55.920650+0000","last_peered":"2024-09-11T01:27:55.920650+0000","last_clean":"2024-09-11T01:27:55.920650+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:27:55.920650+0000","last_undegraded":"2024-09-11T01:27:55.920650+0000","last_fullsized":"2024-09-11T01:27:55.920650+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20957,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921309+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:27:55.921309+0000","last_peered":"2024-09-11T01:27:55.921309+0000","last_clean":"2024-09-11T01:27:55.921309+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:27:55.921309+0000","last_undegraded":"2024-09-11T01:27:55.921309+0000","last_fullsized":"2024-09-11T01:27:55.921309+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20930,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920811+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:27:55.920811+0000","last_peered":"2024-09-11T01:27:55.920811+0000","last_clean":"2024-09-11T01:27:55.920811+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:27:55.920811+0000","last_undegraded":"2024-09-11T01:27:55.920811+0000","last_fullsized":"2024-09-11T01:27:55.920811+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921381+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:27:55.921381+0000","last_peered":"2024-09-11T01:27:55.921381+0000","last_clean":"2024-09-11T01:27:55.921381+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:27:55.921381+0000","last_undegraded":"2024-09-11T01:27:55.921381+0000","last_fullsized":"2024-09-11T01:27:55.921381+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920417+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:27:55.920417+0000","last_peered":"2024-09-11T01:27:55.920417+0000","last_clean":"2024-09-11T01:27:55.920417+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:27:55.920417+0000","last_undegraded":"2024-09-11T01:27:55.920417+0000","last_fullsized":"2024-09-11T01:27:55.920417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20415,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.920606+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:27:55.920606+0000","last_peered":"2024-09-11T01:27:55.920606+0000","last_clean":"2024-09-11T01:27:55.920606+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:27:55.920606+0000","last_undegraded":"2024-09-11T01:27:55.920606+0000","last_fullsized":"2024-09-11T01:27:55.920606+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49608,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66977792,"data_stored":66898306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790107,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830112,"statfs":{"total":96636764160,"available":96082034688,"internally_reserved":0,"allocated":540672,"data_stored":173369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:18.112 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:28:18.112 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'20594", 'reported_seq': 28926, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:28:14.104500+0000', 'last_change': '2024-09-11T01:27:54.654949+0000', 'last_active': '2024-09-11T01:28:14.104500+0000', 'last_peered': '2024-09-11T01:28:14.104500+0000', 'last_clean': '2024-09-11T01:15:17.947972+0000', 'last_became_active': '2024-09-11T01:15:20.541256+0000', 'last_became_peered': '2024-09-11T01:15:20.541256+0000', 'last_unstale': '2024-09-11T01:28:14.104500+0000', 'last_undegraded': '2024-09-11T01:15:20.101344+0000', 'last_fullsized': '2024-09-11T01:15:19.986754+0000', 'mapping_epoch': 38, 'log_start': "41'10500", 'ondisk_log_start': "41'10500", 'created': 16, 'last_epoch_clean': 35, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "32'3681", 'last_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:13:45.564293+0000', 'last_clean_scrub_stamp': '2024-09-11T01:15:10.272127+0000', 'objects_scrubbed': 3628, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1481, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 26, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16958, 'num_write_kb': 26660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4081, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -26}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:28:18.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:28:18.482 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:18.482 INFO:teuthology.orchestra.run.smithi039.stdout:{"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26/24 objects degraded (108.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":898,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017317,"num_in_osds":3,"osd_in_since":1726017311,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1431085056,"bytes_avail":288479207424,"bytes_total":289910292480,"degraded_objects":26,"degraded_total":24,"degraded_ratio":1.0833333333333333,"recovering_objects_per_sec":263,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2107,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:13:19:552800+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2024-09-11T01:22:08.591865+0000","services":{}},"progress_events":{}} 2024-09-11T01:28:18.495 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-11T01:28:18.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:18.789 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:18.790 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:18.802 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":506,"stamp":"2024-09-11T01:28:18.694707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49609,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1401728,"kb_used_data":66936,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281713792,"statfs":{"total":289910292480,"available":288474923008,"internally_reserved":0,"allocated":68542464,"data_stored":67419117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1120,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1120,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001552"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28605,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921440+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:27:55.921440+0000","last_peered":"2024-09-11T01:27:55.921440+0000","last_clean":"2024-09-11T01:27:55.921440+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:27:55.921440+0000","last_undegraded":"2024-09-11T01:27:55.921440+0000","last_fullsized":"2024-09-11T01:27:55.921440+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20438,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.922280+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:27:55.922280+0000","last_peered":"2024-09-11T01:27:55.922280+0000","last_clean":"2024-09-11T01:27:55.922280+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:27:55.922280+0000","last_undegraded":"2024-09-11T01:27:55.922280+0000","last_fullsized":"2024-09-11T01:27:55.922280+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20583,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921121+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:27:55.921121+0000","last_peered":"2024-09-11T01:27:55.921121+0000","last_clean":"2024-09-11T01:27:55.921121+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:27:55.921121+0000","last_undegraded":"2024-09-11T01:27:55.921121+0000","last_fullsized":"2024-09-11T01:27:55.921121+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28926,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:28:14.104500+0000","last_change":"2024-09-11T01:27:54.654949+0000","last_active":"2024-09-11T01:28:14.104500+0000","last_peered":"2024-09-11T01:28:14.104500+0000","last_clean":"2024-09-11T01:15:17.947972+0000","last_became_active":"2024-09-11T01:15:20.541256+0000","last_became_peered":"2024-09-11T01:15:20.541256+0000","last_unstale":"2024-09-11T01:28:14.104500+0000","last_undegraded":"2024-09-11T01:15:20.101344+0000","last_fullsized":"2024-09-11T01:15:19.986754+0000","mapping_epoch":38,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4081,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-26}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:27:55.921578+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:27:55.921578+0000","last_peered":"2024-09-11T01:27:55.921578+0000","last_clean":"2024-09-11T01:27:55.921578+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:27:55.921578+0000","last_undegraded":"2024-09-11T01:27:55.921578+0000","last_fullsized":"2024-09-11T01:27:55.921578+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687892+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687892+0000","last_peered":"2024-09-11T01:28:15.687892+0000","last_clean":"2024-09-11T01:28:15.687892+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687892+0000","last_undegraded":"2024-09-11T01:28:15.687892+0000","last_fullsized":"2024-09-11T01:28:15.687892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49608,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":66977792,"data_stored":66898306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508444,"kb_used_data":65860,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93863396,"statfs":{"total":96636764160,"available":96116117504,"internally_reserved":0,"allocated":67440640,"data_stored":67062498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66928640,"data_stored":66892510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:21.804 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:22.115 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:22.115 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:22.127 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T01:28:20.695069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399352,"kb_used_data":64568,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281716168,"statfs":{"total":289910292480,"available":288477356032,"internally_reserved":0,"allocated":66117632,"data_stored":64983334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001621"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687892+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687892+0000","last_peered":"2024-09-11T01:28:15.687892+0000","last_clean":"2024-09-11T01:28:15.687892+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687892+0000","last_undegraded":"2024-09-11T01:28:15.687892+0000","last_fullsized":"2024-09-11T01:28:15.687892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64457090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506068,"kb_used_data":63492,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93865772,"statfs":{"total":96636764160,"available":96118550528,"internally_reserved":0,"allocated":65015808,"data_stored":64626715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64487424,"data_stored":64451294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:22.129 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-11T01:28:22.129 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-11T01:28:22.129 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:22.424 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:22.424 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:22.436 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T01:28:20.695069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399352,"kb_used_data":64568,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281716168,"statfs":{"total":289910292480,"available":288477356032,"internally_reserved":0,"allocated":66117632,"data_stored":64983334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1146,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001621"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687892+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687892+0000","last_peered":"2024-09-11T01:28:15.687892+0000","last_clean":"2024-09-11T01:28:15.687892+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687892+0000","last_undegraded":"2024-09-11T01:28:15.687892+0000","last_fullsized":"2024-09-11T01:28:15.687892+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64457090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790108,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506068,"kb_used_data":63492,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93865772,"statfs":{"total":96636764160,"available":96118550528,"internally_reserved":0,"allocated":65015808,"data_stored":64626715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64487424,"data_stored":64451294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:22.438 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:22.731 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:22.732 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:22.744 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-11T01:28:22.695371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399352,"kb_used_data":64568,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281716168,"statfs":{"total":289910292480,"available":288477356032,"internally_reserved":0,"allocated":66117632,"data_stored":64983334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.040292"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687937+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687937+0000","last_peered":"2024-09-11T01:28:15.687937+0000","last_clean":"2024-09-11T01:28:15.687937+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687937+0000","last_undegraded":"2024-09-11T01:28:15.687937+0000","last_fullsized":"2024-09-11T01:28:15.687937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64457090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506068,"kb_used_data":63492,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93865772,"statfs":{"total":96636764160,"available":96118550528,"internally_reserved":0,"allocated":65015808,"data_stored":64626715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64487424,"data_stored":64451294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:22.745 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-11T01:28:22.746 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T01:28:22.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T01:28:22.757 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:23.052 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:23.053 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:23.065 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-11T01:28:22.695371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399352,"kb_used_data":64568,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281716168,"statfs":{"total":289910292480,"available":288477356032,"internally_reserved":0,"allocated":66117632,"data_stored":64983334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.040292"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687937+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687937+0000","last_peered":"2024-09-11T01:28:15.687937+0000","last_clean":"2024-09-11T01:28:15.687937+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687937+0000","last_undegraded":"2024-09-11T01:28:15.687937+0000","last_fullsized":"2024-09-11T01:28:15.687937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64457090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506068,"kb_used_data":63492,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93865772,"statfs":{"total":96636764160,"available":96118550528,"internally_reserved":0,"allocated":65015808,"data_stored":64626715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64487424,"data_stored":64451294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:23.066 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:23.364 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:23.365 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:23.377 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-11T01:28:22.695371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399352,"kb_used_data":64568,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281716168,"statfs":{"total":289910292480,"available":288477356032,"internally_reserved":0,"allocated":66117632,"data_stored":64983334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.040292"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687937+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687937+0000","last_peered":"2024-09-11T01:28:15.687937+0000","last_clean":"2024-09-11T01:28:15.687937+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687937+0000","last_undegraded":"2024-09-11T01:28:15.687937+0000","last_fullsized":"2024-09-11T01:28:15.687937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64457090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506068,"kb_used_data":63492,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93865772,"statfs":{"total":96636764160,"available":96118550528,"internally_reserved":0,"allocated":65015808,"data_stored":64626715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64487424,"data_stored":64451294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:23.378 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T01:28:23.378 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:23.678 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:23.678 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:23.690 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-11T01:28:22.695371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1399352,"kb_used_data":64568,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281716168,"statfs":{"total":289910292480,"available":288477356032,"internally_reserved":0,"allocated":66117632,"data_stored":64983334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.040292"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687937+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687937+0000","last_peered":"2024-09-11T01:28:15.687937+0000","last_clean":"2024-09-11T01:28:15.687937+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687937+0000","last_undegraded":"2024-09-11T01:28:15.687937+0000","last_fullsized":"2024-09-11T01:28:15.687937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":64536576,"data_stored":64457090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506068,"kb_used_data":63492,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93865772,"statfs":{"total":96636764160,"available":96118550528,"internally_reserved":0,"allocated":65015808,"data_stored":64626715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64487424,"data_stored":64451294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:23.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:28:23.990 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:23.990 INFO:teuthology.orchestra.run.smithi039.stdout:{"epoch":63,"fsid":"bcb624ec-d02e-4f87-a117-bb3a3b752391","created":"2024-09-11T01:13:19.553161+0000","modified":"2024-09-11T01:28:15.651638+0000","last_up_change":"2024-09-11T01:15:17.956478+0000","last_in_change":"2024-09-11T01:15:11.886947+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:13:35.284028+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-11T01:13:44.644516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f5f0de43-07f1-4db0-9ad7-4564c8fd2953","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":21,"up_from":25,"up_thru":62,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":2566795661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6801","nonce":2566795661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6803","nonce":2566795661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":2566795661}]},"public_addr":"172.21.15.39:6800/2566795661","cluster_addr":"172.21.15.39:6801/2566795661","heartbeat_back_addr":"172.21.15.39:6803/2566795661","heartbeat_front_addr":"172.21.15.39:6802/2566795661","state":["exists","up"]},{"osd":1,"uuid":"5df6a606-9f1b-4a88-8879-e1661e7956c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":26,"up_from":37,"up_thru":60,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":1018236417}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6809","nonce":1018236417}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6811","nonce":1018236417}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":1018236417}]},"public_addr":"172.21.15.39:6808/1018236417","cluster_addr":"172.21.15.39:6809/1018236417","heartbeat_back_addr":"172.21.15.39:6811/1018236417","heartbeat_front_addr":"172.21.15.39:6810/1018236417","state":["exists","up"]},{"osd":2,"uuid":"ee7baf02-576b-4245-9572-bfd1ab1e3a0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":51,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":833967465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6805","nonce":833967465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6807","nonce":833967465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":833967465}]},"public_addr":"172.21.15.39:6804/833967465","cluster_addr":"172.21.15.39:6805/833967465","heartbeat_back_addr":"172.21.15.39:6807/833967465","heartbeat_front_addr":"172.21.15.39:6806/833967465","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.158089+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.419012+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:13:27.137433+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:28:25.003 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T01:28:25.004 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T01:28:25.147 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:28:25.147 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T01:28:25.147 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:28:25.158 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T01:28:25.458 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T01:28:25.471 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T01:28:25.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T01:28:25.597 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:28:25.597 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T01:28:25.597 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:28:25.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T01:28:25.904 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T01:28:25.918 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T01:28:25.918 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T01:28:26.043 INFO:teuthology.orchestra.run.smithi039.stdout:{ 2024-09-11T01:28:26.043 INFO:teuthology.orchestra.run.smithi039.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T01:28:26.043 INFO:teuthology.orchestra.run.smithi039.stdout:} 2024-09-11T01:28:26.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T01:28:26.354 INFO:teuthology.orchestra.run.smithi039.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T01:28:26.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:26.662 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:26.663 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:26.674 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-11T01:28:24.695743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1396952,"kb_used_data":62168,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281718568,"statfs":{"total":289910292480,"available":288479813632,"internally_reserved":0,"allocated":63660032,"data_stored":62525734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1366739975},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001871"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28609,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682408+0000","last_change":"2024-09-11T01:25:02.908550+0000","last_active":"2024-09-11T01:28:15.682408+0000","last_peered":"2024-09-11T01:28:15.682408+0000","last_clean":"2024-09-11T01:28:15.682408+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:15.682408+0000","last_undegraded":"2024-09-11T01:28:15.682408+0000","last_fullsized":"2024-09-11T01:28:15.682408+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:56.835898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'20522","reported_seq":20646,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688618+0000","last_change":"2024-09-11T01:14:48.907391+0000","last_active":"2024-09-11T01:28:15.688618+0000","last_peered":"2024-09-11T01:28:15.688618+0000","last_clean":"2024-09-11T01:28:15.688618+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:15.688618+0000","last_undegraded":"2024-09-11T01:28:15.688618+0000","last_fullsized":"2024-09-11T01:28:15.688618+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1924","last_scrub_stamp":"2024-09-11T01:14:48.907228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:48.907228+0000","objects_scrubbed":1898,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:20.336893+0000","scrub_duration":632,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20253,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687798+0000","last_change":"2024-09-11T01:26:19.903190+0000","last_active":"2024-09-11T01:28:15.687798+0000","last_peered":"2024-09-11T01:28:15.687798+0000","last_clean":"2024-09-11T01:28:15.687798+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:15.687798+0000","last_undegraded":"2024-09-11T01:28:15.687798+0000","last_fullsized":"2024-09-11T01:28:15.687798+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'1981","last_scrub_stamp":"2024-09-11T01:14:50.130952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:50.130952+0000","objects_scrubbed":1935,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:40.350009+0000","scrub_duration":841,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688312+0000","last_change":"2024-09-11T01:14:27.028034+0000","last_active":"2024-09-11T01:28:15.688312+0000","last_peered":"2024-09-11T01:28:15.688312+0000","last_clean":"2024-09-11T01:28:15.688312+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:15.688312+0000","last_undegraded":"2024-09-11T01:28:15.688312+0000","last_fullsized":"2024-09-11T01:28:15.688312+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:13:45.564293+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:05.610054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'20277","reported_seq":20442,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682148+0000","last_change":"2024-09-11T01:15:04.311026+0000","last_active":"2024-09-11T01:28:15.682148+0000","last_peered":"2024-09-11T01:28:15.682148+0000","last_clean":"2024-09-11T01:28:15.682148+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:15.682148+0000","last_undegraded":"2024-09-11T01:28:15.682148+0000","last_fullsized":"2024-09-11T01:28:15.682148+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3252","last_scrub_stamp":"2024-09-11T01:15:04.310805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:04.310805+0000","objects_scrubbed":3196,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:18.718777+0000","scrub_duration":1512,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'20207","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687482+0000","last_change":"2024-09-11T01:27:38.964907+0000","last_active":"2024-09-11T01:28:15.687482+0000","last_peered":"2024-09-11T01:28:15.687482+0000","last_clean":"2024-09-11T01:28:15.687482+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:15.687482+0000","last_undegraded":"2024-09-11T01:28:15.687482+0000","last_fullsized":"2024-09-11T01:28:15.687482+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3559","last_scrub_stamp":"2024-09-11T01:15:08.270496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:08.270496+0000","objects_scrubbed":3508,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:03:06.066680+0000","scrub_duration":1435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"45'20812","reported_seq":21122,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687325+0000","last_change":"2024-09-11T01:27:56.017930+0000","last_active":"2024-09-11T01:28:15.687325+0000","last_peered":"2024-09-11T01:28:15.687325+0000","last_clean":"2024-09-11T01:28:15.687325+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:15.687325+0000","last_undegraded":"2024-09-11T01:28:15.687325+0000","last_fullsized":"2024-09-11T01:28:15.687325+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3306","last_scrub_stamp":"2024-09-11T01:15:03.099483+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:03.099483+0000","objects_scrubbed":3243,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:19:22.346168+0000","scrub_duration":1279,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20587,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.681985+0000","last_change":"2024-09-11T01:15:02.026277+0000","last_active":"2024-09-11T01:28:15.681985+0000","last_peered":"2024-09-11T01:28:15.681985+0000","last_clean":"2024-09-11T01:28:15.681985+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:15.681985+0000","last_undegraded":"2024-09-11T01:28:15.681985+0000","last_fullsized":"2024-09-11T01:28:15.681985+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3106","last_scrub_stamp":"2024-09-11T01:15:02.026177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:02.026177+0000","objects_scrubbed":3045,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:31.320133+0000","scrub_duration":1220,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687232+0000","last_change":"2024-09-11T01:15:19.992819+0000","last_active":"2024-09-11T01:28:15.687232+0000","last_peered":"2024-09-11T01:28:15.687232+0000","last_clean":"2024-09-11T01:28:15.687232+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:15.687232+0000","last_undegraded":"2024-09-11T01:28:15.687232+0000","last_fullsized":"2024-09-11T01:28:15.687232+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:13.977719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'20249","reported_seq":28636,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687573+0000","last_change":"2024-09-11T01:17:37.909823+0000","last_active":"2024-09-11T01:28:15.687573+0000","last_peered":"2024-09-11T01:28:15.687573+0000","last_clean":"2024-09-11T01:28:15.687573+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:15.687573+0000","last_undegraded":"2024-09-11T01:28:15.687573+0000","last_fullsized":"2024-09-11T01:28:15.687573+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2163","last_scrub_stamp":"2024-09-11T01:14:51.929170+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:51.929170+0000","objects_scrubbed":2127,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:48:30.134264+0000","scrub_duration":703,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688278+0000","last_change":"2024-09-11T01:14:27.026877+0000","last_active":"2024-09-11T01:28:15.688278+0000","last_peered":"2024-09-11T01:28:15.688278+0000","last_clean":"2024-09-11T01:28:15.688278+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:15.688278+0000","last_undegraded":"2024-09-11T01:28:15.688278+0000","last_fullsized":"2024-09-11T01:28:15.688278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:38.703308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28989,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.744409+0000","last_change":"2024-09-11T01:28:15.744305+0000","last_active":"2024-09-11T01:28:15.744409+0000","last_peered":"2024-09-11T01:28:15.744409+0000","last_clean":"2024-09-11T01:28:15.744409+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:15.744409+0000","last_undegraded":"2024-09-11T01:28:15.744409+0000","last_fullsized":"2024-09-11T01:28:15.744409+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3681","last_scrub_stamp":"2024-09-11T01:15:10.272127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:10.272127+0000","objects_scrubbed":3628,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:06.904862+0000","scrub_duration":1481,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.682012+0000","last_change":"2024-09-11T01:15:20.104241+0000","last_active":"2024-09-11T01:28:15.682012+0000","last_peered":"2024-09-11T01:28:15.682012+0000","last_clean":"2024-09-11T01:28:15.682012+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:15.682012+0000","last_undegraded":"2024-09-11T01:28:15.682012+0000","last_fullsized":"2024-09-11T01:28:15.682012+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:43.434834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'20139","reported_seq":28340,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688307+0000","last_change":"2024-09-11T01:23:46.889777+0000","last_active":"2024-09-11T01:28:15.688307+0000","last_peered":"2024-09-11T01:28:15.688307+0000","last_clean":"2024-09-11T01:28:15.688307+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:15.688307+0000","last_undegraded":"2024-09-11T01:28:15.688307+0000","last_fullsized":"2024-09-11T01:28:15.688307+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2276","last_scrub_stamp":"2024-09-11T01:14:53.159379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:53.159379+0000","objects_scrubbed":2227,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:57.456418+0000","scrub_duration":885,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687465+0000","last_change":"2024-09-11T01:15:20.047138+0000","last_active":"2024-09-11T01:28:15.687465+0000","last_peered":"2024-09-11T01:28:15.687465+0000","last_clean":"2024-09-11T01:28:15.687465+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:15.687465+0000","last_undegraded":"2024-09-11T01:28:15.687465+0000","last_fullsized":"2024-09-11T01:28:15.687465+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:39.316856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29065,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688189+0000","last_change":"2024-09-11T01:25:28.008212+0000","last_active":"2024-09-11T01:28:15.688189+0000","last_peered":"2024-09-11T01:28:15.688189+0000","last_clean":"2024-09-11T01:28:15.688189+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:15.688189+0000","last_undegraded":"2024-09-11T01:28:15.688189+0000","last_fullsized":"2024-09-11T01:28:15.688189+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'2404","last_scrub_stamp":"2024-09-11T01:14:54.063207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:14:54.063207+0000","objects_scrubbed":2357,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:12.120018+0000","scrub_duration":785,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28866,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687991+0000","last_change":"2024-09-11T01:20:45.713425+0000","last_active":"2024-09-11T01:28:15.687991+0000","last_peered":"2024-09-11T01:28:15.687991+0000","last_clean":"2024-09-11T01:28:15.687991+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:15.687991+0000","last_undegraded":"2024-09-11T01:28:15.687991+0000","last_fullsized":"2024-09-11T01:28:15.687991+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3367","last_scrub_stamp":"2024-09-11T01:15:05.048814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.048814+0000","objects_scrubbed":3308,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:30.835865+0000","scrub_duration":1209,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687619+0000","last_change":"2024-09-11T01:15:20.197782+0000","last_active":"2024-09-11T01:28:15.687619+0000","last_peered":"2024-09-11T01:28:15.687619+0000","last_clean":"2024-09-11T01:28:15.687619+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:15.687619+0000","last_undegraded":"2024-09-11T01:28:15.687619+0000","last_fullsized":"2024-09-11T01:28:15.687619+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:27.359097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20961,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688080+0000","last_change":"2024-09-11T01:27:11.864950+0000","last_active":"2024-09-11T01:28:15.688080+0000","last_peered":"2024-09-11T01:28:15.688080+0000","last_clean":"2024-09-11T01:28:15.688080+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:15.688080+0000","last_undegraded":"2024-09-11T01:28:15.688080+0000","last_fullsized":"2024-09-11T01:28:15.688080+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3663","last_scrub_stamp":"2024-09-11T01:15:09.244556+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:09.244556+0000","objects_scrubbed":3599,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:25:10.755769+0000","scrub_duration":1410,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.688594+0000","last_change":"2024-09-11T01:14:27.028529+0000","last_active":"2024-09-11T01:28:15.688594+0000","last_peered":"2024-09-11T01:28:15.688594+0000","last_clean":"2024-09-11T01:28:15.688594+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:15.688594+0000","last_undegraded":"2024-09-11T01:28:15.688594+0000","last_fullsized":"2024-09-11T01:28:15.688594+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:03.360254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20934,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687837+0000","last_change":"2024-09-11T01:26:48.137401+0000","last_active":"2024-09-11T01:28:15.687837+0000","last_peered":"2024-09-11T01:28:15.687837+0000","last_clean":"2024-09-11T01:28:15.687837+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:15.687837+0000","last_undegraded":"2024-09-11T01:28:15.687837+0000","last_fullsized":"2024-09-11T01:28:15.687837+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3518","last_scrub_stamp":"2024-09-11T01:15:07.160741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:07.160741+0000","objects_scrubbed":3461,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:18:35.252647+0000","scrub_duration":1304,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687969+0000","last_change":"2024-09-11T01:15:20.198626+0000","last_active":"2024-09-11T01:28:15.687969+0000","last_peered":"2024-09-11T01:28:15.687969+0000","last_clean":"2024-09-11T01:28:15.687969+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:15.687969+0000","last_undegraded":"2024-09-11T01:28:15.687969+0000","last_fullsized":"2024-09-11T01:28:15.687969+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:14:34.304679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687937+0000","last_change":"2024-09-11T01:15:19.996024+0000","last_active":"2024-09-11T01:28:15.687937+0000","last_peered":"2024-09-11T01:28:15.687937+0000","last_clean":"2024-09-11T01:28:15.687937+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:15.687937+0000","last_undegraded":"2024-09-11T01:28:15.687937+0000","last_fullsized":"2024-09-11T01:28:15.687937+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:35.453339+0000","last_clean_scrub_stamp":"2024-09-11T01:13:35.453339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:50.074737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'20138","reported_seq":20419,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:15.687570+0000","last_change":"2024-09-11T01:25:55.265523+0000","last_active":"2024-09-11T01:28:15.687570+0000","last_peered":"2024-09-11T01:28:15.687570+0000","last_clean":"2024-09-11T01:28:15.687570+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:15.687570+0000","last_undegraded":"2024-09-11T01:28:15.687570+0000","last_fullsized":"2024-09-11T01:28:15.687570+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'3390","last_scrub_stamp":"2024-09-11T01:15:05.985745+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:13:45.564293+0000","last_clean_scrub_stamp":"2024-09-11T01:15:05.985745+0000","objects_scrubbed":3333,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:26:06.965237+0000","scrub_duration":1094,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":62078976,"data_stored":61999490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347372,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024468,"statfs":{"total":96636764160,"available":96281055232,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790109,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545912,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825928,"statfs":{"total":96636764160,"available":96077750272,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503668,"kb_used_data":61092,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93868172,"statfs":{"total":96636764160,"available":96121008128,"internally_reserved":0,"allocated":62558208,"data_stored":62169115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":453177002},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":62029824,"data_stored":61993694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-11T01:13:45.564293+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-11T01:14:48.907228+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=14, tm_sec=48, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-11T01:14:50.130952+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=14, tm_sec=50, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-11T01:13:45.564293+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=45, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-11T01:15:04.310805+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=4, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-11T01:15:08.270496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=8, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.679 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-11T01:15:03.099483+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=3, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-11T01:15:02.026177+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=2, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-11T01:14:51.929170+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=14, tm_sec=51, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-11T01:15:10.272127+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=10, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-11T01:14:53.159379+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=14, tm_sec=53, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.680 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-11T01:14:54.063207+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=14, tm_sec=54, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-11T01:15:05.048814+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=5, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-11T01:15:09.244556+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=9, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-11T01:15:07.160741+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=7, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-11T01:13:35.453339+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=35, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-11T01:15:05.985745+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=15, tm_sec=5, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=28, tm_sec=23, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:28:26.681 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T01:28:46.682 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:28:46.992 INFO:teuthology.orchestra.run.smithi039.stdout: 2024-09-11T01:28:46.993 INFO:teuthology.orchestra.run.smithi039.stderr:dumped all 2024-09-11T01:28:47.005 INFO:teuthology.orchestra.run.smithi039.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-11T01:28:46.700307+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285233,"num_write_kb":488751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49635,"num_bytes_recovered":36907877,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1391496,"kb_used_data":52568,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281724024,"statfs":{"total":289910292480,"available":288485400576,"internally_reserved":0,"allocated":53829632,"data_stored":52698947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13305,"internal_metadata":1370999815},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002599"},"pg_stats":[{"pgid":"2.f","version":"45'20238","reported_seq":28617,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:27.882200+0000","last_change":"2024-09-11T01:28:27.882200+0000","last_active":"2024-09-11T01:28:27.882200+0000","last_peered":"2024-09-11T01:28:27.882200+0000","last_clean":"2024-09-11T01:28:27.882200+0000","last_became_active":"2024-09-11T01:25:02.844118+0000","last_became_peered":"2024-09-11T01:25:02.844118+0000","last_unstale":"2024-09-11T01:28:27.882200+0000","last_undegraded":"2024-09-11T01:28:27.882200+0000","last_fullsized":"2024-09-11T01:28:27.882200+0000","mapping_epoch":46,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20238","last_scrub_stamp":"2024-09-11T01:28:27.882067+0000","last_deep_scrub":"45'20238","last_deep_scrub_stamp":"2024-09-11T01:28:27.882067+0000","last_clean_scrub_stamp":"2024-09-11T01:28:27.882067+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:39.092416+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20238,"num_write_kb":40476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"63'20524","reported_seq":20656,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:31.977252+0000","last_change":"2024-09-11T01:28:31.977252+0000","last_active":"2024-09-11T01:28:31.977252+0000","last_peered":"2024-09-11T01:28:31.977252+0000","last_clean":"2024-09-11T01:28:31.977252+0000","last_became_active":"2024-09-11T01:14:27.027594+0000","last_became_peered":"2024-09-11T01:14:27.027594+0000","last_unstale":"2024-09-11T01:28:31.977252+0000","last_undegraded":"2024-09-11T01:28:31.977252+0000","last_fullsized":"2024-09-11T01:28:31.977252+0000","mapping_epoch":31,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20524","last_scrub_stamp":"2024-09-11T01:28:31.977144+0000","last_deep_scrub":"63'20524","last_deep_scrub_stamp":"2024-09-11T01:28:31.977144+0000","last_clean_scrub_stamp":"2024-09-11T01:28:31.977144+0000","objects_scrubbed":2,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:07.259644+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18620,"num_write_kb":33448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'20046","reported_seq":20262,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:38.436523+0000","last_change":"2024-09-11T01:28:38.436523+0000","last_active":"2024-09-11T01:28:38.436523+0000","last_peered":"2024-09-11T01:28:38.436523+0000","last_clean":"2024-09-11T01:28:38.436523+0000","last_became_active":"2024-09-11T01:26:19.818012+0000","last_became_peered":"2024-09-11T01:26:19.818012+0000","last_unstale":"2024-09-11T01:28:38.436523+0000","last_undegraded":"2024-09-11T01:28:38.436523+0000","last_fullsized":"2024-09-11T01:28:38.436523+0000","mapping_epoch":52,"log_start":"41'10000","ondisk_log_start":"41'10000","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20046","last_scrub_stamp":"2024-09-11T01:28:38.436415+0000","last_deep_scrub":"45'20046","last_deep_scrub_stamp":"2024-09-11T01:28:38.436415+0000","last_clean_scrub_stamp":"2024-09-11T01:28:38.436415+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:54.956164+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18107,"num_write_kb":32341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4007,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"45'20362","reported_seq":20459,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:30.996731+0000","last_change":"2024-09-11T01:28:30.996731+0000","last_active":"2024-09-11T01:28:30.996731+0000","last_peered":"2024-09-11T01:28:30.996731+0000","last_clean":"2024-09-11T01:28:30.996731+0000","last_became_active":"2024-09-11T01:14:27.027708+0000","last_became_peered":"2024-09-11T01:14:27.027708+0000","last_unstale":"2024-09-11T01:28:30.996731+0000","last_undegraded":"2024-09-11T01:28:30.996731+0000","last_fullsized":"2024-09-11T01:28:30.996731+0000","mapping_epoch":31,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20362","last_scrub_stamp":"2024-09-11T01:28:30.996595+0000","last_deep_scrub":"45'20362","last_deep_scrub_stamp":"2024-09-11T01:28:30.996595+0000","last_clean_scrub_stamp":"2024-09-11T01:28:30.996595+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:41:50.855790+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20362,"num_write_kb":40724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"63'20278","reported_seq":20451,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:28.926281+0000","last_change":"2024-09-11T01:28:28.926281+0000","last_active":"2024-09-11T01:28:28.926281+0000","last_peered":"2024-09-11T01:28:28.926281+0000","last_clean":"2024-09-11T01:28:28.926281+0000","last_became_active":"2024-09-11T01:14:27.029203+0000","last_became_peered":"2024-09-11T01:14:27.029203+0000","last_unstale":"2024-09-11T01:28:28.926281+0000","last_undegraded":"2024-09-11T01:28:28.926281+0000","last_fullsized":"2024-09-11T01:28:28.926281+0000","mapping_epoch":31,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20278","last_scrub_stamp":"2024-09-11T01:28:28.926203+0000","last_deep_scrub":"63'20278","last_deep_scrub_stamp":"2024-09-11T01:28:28.926203+0000","last_clean_scrub_stamp":"2024-09-11T01:28:28.926203+0000","objects_scrubbed":1,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:53.642072+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17075,"num_write_kb":27760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"63'20208","reported_seq":20503,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:36.499913+0000","last_change":"2024-09-11T01:28:36.499913+0000","last_active":"2024-09-11T01:28:36.499913+0000","last_peered":"2024-09-11T01:28:36.499913+0000","last_clean":"2024-09-11T01:28:36.499913+0000","last_became_active":"2024-09-11T01:27:38.857374+0000","last_became_peered":"2024-09-11T01:27:38.857374+0000","last_unstale":"2024-09-11T01:28:36.499913+0000","last_undegraded":"2024-09-11T01:28:36.499913+0000","last_fullsized":"2024-09-11T01:28:36.499913+0000","mapping_epoch":58,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20208","last_scrub_stamp":"2024-09-11T01:28:36.499827+0000","last_deep_scrub":"63'20208","last_deep_scrub_stamp":"2024-09-11T01:28:36.499827+0000","last_clean_scrub_stamp":"2024-09-11T01:28:36.499827+0000","objects_scrubbed":1,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:27.600421+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16695,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"63'20814","reported_seq":21132,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:34.530204+0000","last_change":"2024-09-11T01:28:34.530204+0000","last_active":"2024-09-11T01:28:34.530204+0000","last_peered":"2024-09-11T01:28:34.530204+0000","last_clean":"2024-09-11T01:28:34.530204+0000","last_became_active":"2024-09-11T01:27:55.922688+0000","last_became_peered":"2024-09-11T01:27:55.922688+0000","last_unstale":"2024-09-11T01:28:34.530204+0000","last_undegraded":"2024-09-11T01:28:34.530204+0000","last_fullsized":"2024-09-11T01:28:34.530204+0000","mapping_epoch":60,"log_start":"41'10800","ondisk_log_start":"41'10800","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20814","last_scrub_stamp":"2024-09-11T01:28:34.530121+0000","last_deep_scrub":"63'20814","last_deep_scrub_stamp":"2024-09-11T01:28:34.530121+0000","last_clean_scrub_stamp":"2024-09-11T01:28:34.530121+0000","objects_scrubbed":2,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:54.842538+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17565,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4283,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"45'20424","reported_seq":20595,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:26.868850+0000","last_change":"2024-09-11T01:28:26.868850+0000","last_active":"2024-09-11T01:28:26.868850+0000","last_peered":"2024-09-11T01:28:26.868850+0000","last_clean":"2024-09-11T01:28:26.868850+0000","last_became_active":"2024-09-11T01:14:27.028253+0000","last_became_peered":"2024-09-11T01:14:27.028253+0000","last_unstale":"2024-09-11T01:28:26.868850+0000","last_undegraded":"2024-09-11T01:28:26.868850+0000","last_fullsized":"2024-09-11T01:28:26.868850+0000","mapping_epoch":31,"log_start":"41'10400","ondisk_log_start":"41'10400","created":16,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20424","last_scrub_stamp":"2024-09-11T01:28:26.868754+0000","last_deep_scrub":"45'20424","last_deep_scrub_stamp":"2024-09-11T01:28:26.868754+0000","last_clean_scrub_stamp":"2024-09-11T01:28:26.868754+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:25.917701+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17377,"num_write_kb":28664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:26.672417+0000","last_change":"2024-09-11T01:28:26.672417+0000","last_active":"2024-09-11T01:28:26.672417+0000","last_peered":"2024-09-11T01:28:26.672417+0000","last_clean":"2024-09-11T01:28:26.672417+0000","last_became_active":"2024-09-11T01:15:19.992125+0000","last_became_peered":"2024-09-11T01:15:19.992125+0000","last_unstale":"2024-09-11T01:28:26.672417+0000","last_undegraded":"2024-09-11T01:28:26.672417+0000","last_fullsized":"2024-09-11T01:28:26.672417+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:28:26.672346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:28:26.672346+0000","last_clean_scrub_stamp":"2024-09-11T01:28:26.672346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:34.065232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010990539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"63'20250","reported_seq":28645,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:30.558947+0000","last_change":"2024-09-11T01:28:30.558947+0000","last_active":"2024-09-11T01:28:30.558947+0000","last_peered":"2024-09-11T01:28:30.558947+0000","last_clean":"2024-09-11T01:28:30.558947+0000","last_became_active":"2024-09-11T01:17:37.901627+0000","last_became_peered":"2024-09-11T01:17:37.901627+0000","last_unstale":"2024-09-11T01:28:30.558947+0000","last_undegraded":"2024-09-11T01:28:30.558947+0000","last_fullsized":"2024-09-11T01:28:30.558947+0000","mapping_epoch":40,"log_start":"41'10200","ondisk_log_start":"41'10200","created":16,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20250","last_scrub_stamp":"2024-09-11T01:28:30.558860+0000","last_deep_scrub":"63'20250","last_deep_scrub_stamp":"2024-09-11T01:28:30.558860+0000","last_clean_scrub_stamp":"2024-09-11T01:28:30.558860+0000","objects_scrubbed":1,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:48:56.946085+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18116,"num_write_kb":31980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4125,"num_bytes_recovered":16888291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:26.963101+0000","last_change":"2024-09-11T01:28:26.963101+0000","last_active":"2024-09-11T01:28:26.963101+0000","last_peered":"2024-09-11T01:28:26.963101+0000","last_clean":"2024-09-11T01:28:26.963101+0000","last_became_active":"2024-09-11T01:14:27.026676+0000","last_became_peered":"2024-09-11T01:14:27.026676+0000","last_unstale":"2024-09-11T01:28:26.963101+0000","last_undegraded":"2024-09-11T01:28:26.963101+0000","last_fullsized":"2024-09-11T01:28:26.963101+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:28:26.963043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:28:26.963043+0000","last_clean_scrub_stamp":"2024-09-11T01:28:26.963043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:08.444906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'20594","reported_seq":28997,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:29.946877+0000","last_change":"2024-09-11T01:28:29.946877+0000","last_active":"2024-09-11T01:28:29.946877+0000","last_peered":"2024-09-11T01:28:29.946877+0000","last_clean":"2024-09-11T01:28:29.946877+0000","last_became_active":"2024-09-11T01:28:15.688981+0000","last_became_peered":"2024-09-11T01:28:15.688981+0000","last_unstale":"2024-09-11T01:28:29.946877+0000","last_undegraded":"2024-09-11T01:28:29.946877+0000","last_fullsized":"2024-09-11T01:28:29.946877+0000","mapping_epoch":62,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20594","last_scrub_stamp":"2024-09-11T01:28:29.946780+0000","last_deep_scrub":"45'20594","last_deep_scrub_stamp":"2024-09-11T01:28:29.946780+0000","last_clean_scrub_stamp":"2024-09-11T01:28:29.946780+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:13.011871+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16958,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"63'3","reported_seq":150,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:25.906810+0000","last_change":"2024-09-11T01:28:25.906810+0000","last_active":"2024-09-11T01:28:25.906810+0000","last_peered":"2024-09-11T01:28:25.906810+0000","last_clean":"2024-09-11T01:28:25.906810+0000","last_became_active":"2024-09-11T01:15:20.103835+0000","last_became_peered":"2024-09-11T01:15:20.103835+0000","last_unstale":"2024-09-11T01:28:25.906810+0000","last_undegraded":"2024-09-11T01:28:25.906810+0000","last_fullsized":"2024-09-11T01:28:25.906810+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'3","last_scrub_stamp":"2024-09-11T01:28:25.906752+0000","last_deep_scrub":"63'3","last_deep_scrub_stamp":"2024-09-11T01:28:25.906752+0000","last_clean_scrub_stamp":"2024-09-11T01:28:25.906752+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:17.326689+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00061897500000000002,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"63'20140","reported_seq":28349,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:30.009551+0000","last_change":"2024-09-11T01:28:30.009551+0000","last_active":"2024-09-11T01:28:30.009551+0000","last_peered":"2024-09-11T01:28:30.009551+0000","last_clean":"2024-09-11T01:28:30.009551+0000","last_became_active":"2024-09-11T01:23:46.844207+0000","last_became_peered":"2024-09-11T01:23:46.844207+0000","last_unstale":"2024-09-11T01:28:30.009551+0000","last_undegraded":"2024-09-11T01:28:30.009551+0000","last_fullsized":"2024-09-11T01:28:30.009551+0000","mapping_epoch":44,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20140","last_scrub_stamp":"2024-09-11T01:28:30.009464+0000","last_deep_scrub":"63'20140","last_deep_scrub_stamp":"2024-09-11T01:28:30.009464+0000","last_clean_scrub_stamp":"2024-09-11T01:28:30.009464+0000","objects_scrubbed":1,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:17.785592+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17908,"num_write_kb":31364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4028,"num_bytes_recovered":2957795,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"63'2","reported_seq":137,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:27.651671+0000","last_change":"2024-09-11T01:28:27.651671+0000","last_active":"2024-09-11T01:28:27.651671+0000","last_peered":"2024-09-11T01:28:27.651671+0000","last_clean":"2024-09-11T01:28:27.651671+0000","last_became_active":"2024-09-11T01:15:20.046811+0000","last_became_peered":"2024-09-11T01:15:20.046811+0000","last_unstale":"2024-09-11T01:28:27.651671+0000","last_undegraded":"2024-09-11T01:28:27.651671+0000","last_fullsized":"2024-09-11T01:28:27.651671+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'2","last_scrub_stamp":"2024-09-11T01:28:27.651566+0000","last_deep_scrub":"63'2","last_deep_scrub_stamp":"2024-09-11T01:28:27.651566+0000","last_clean_scrub_stamp":"2024-09-11T01:28:27.651566+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:09:29.370747+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00059519599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'20562","reported_seq":29074,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:27.960208+0000","last_change":"2024-09-11T01:28:27.960208+0000","last_active":"2024-09-11T01:28:27.960208+0000","last_peered":"2024-09-11T01:28:27.960208+0000","last_clean":"2024-09-11T01:28:27.960208+0000","last_became_active":"2024-09-11T01:25:27.933392+0000","last_became_peered":"2024-09-11T01:25:27.933392+0000","last_unstale":"2024-09-11T01:28:27.960208+0000","last_undegraded":"2024-09-11T01:28:27.960208+0000","last_fullsized":"2024-09-11T01:28:27.960208+0000","mapping_epoch":48,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20562","last_scrub_stamp":"2024-09-11T01:28:27.960114+0000","last_deep_scrub":"45'20562","last_deep_scrub_stamp":"2024-09-11T01:28:27.960114+0000","last_clean_scrub_stamp":"2024-09-11T01:28:27.960114+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:48.681930+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18199,"num_write_kb":31684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4179,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'20352","reported_seq":28874,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:35.504085+0000","last_change":"2024-09-11T01:28:35.504085+0000","last_active":"2024-09-11T01:28:35.504085+0000","last_peered":"2024-09-11T01:28:35.504085+0000","last_clean":"2024-09-11T01:28:35.504085+0000","last_became_active":"2024-09-11T01:20:45.684182+0000","last_became_peered":"2024-09-11T01:20:45.684182+0000","last_unstale":"2024-09-11T01:28:35.504085+0000","last_undegraded":"2024-09-11T01:28:35.504085+0000","last_fullsized":"2024-09-11T01:28:35.504085+0000","mapping_epoch":42,"log_start":"41'10300","ondisk_log_start":"41'10300","created":16,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20352","last_scrub_stamp":"2024-09-11T01:28:35.503971+0000","last_deep_scrub":"45'20352","last_deep_scrub_stamp":"2024-09-11T01:28:35.503971+0000","last_clean_scrub_stamp":"2024-09-11T01:28:35.503971+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:56:13.664657+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":27468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4165,"num_bytes_recovered":17055744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:28.619435+0000","last_change":"2024-09-11T01:28:28.619435+0000","last_active":"2024-09-11T01:28:28.619435+0000","last_peered":"2024-09-11T01:28:28.619435+0000","last_clean":"2024-09-11T01:28:28.619435+0000","last_became_active":"2024-09-11T01:15:20.194912+0000","last_became_peered":"2024-09-11T01:15:20.194912+0000","last_unstale":"2024-09-11T01:28:28.619435+0000","last_undegraded":"2024-09-11T01:28:28.619435+0000","last_fullsized":"2024-09-11T01:28:28.619435+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:28:28.619338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:28:28.619338+0000","last_clean_scrub_stamp":"2024-09-11T01:28:28.619338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:08.381587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093865799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'20646","reported_seq":20969,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:37.480922+0000","last_change":"2024-09-11T01:28:37.480922+0000","last_active":"2024-09-11T01:28:37.480922+0000","last_peered":"2024-09-11T01:28:37.480922+0000","last_clean":"2024-09-11T01:28:37.480922+0000","last_became_active":"2024-09-11T01:27:11.763296+0000","last_became_peered":"2024-09-11T01:27:11.763296+0000","last_unstale":"2024-09-11T01:28:37.480922+0000","last_undegraded":"2024-09-11T01:28:37.480922+0000","last_fullsized":"2024-09-11T01:28:37.480922+0000","mapping_epoch":56,"log_start":"41'10600","ondisk_log_start":"41'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20646","last_scrub_stamp":"2024-09-11T01:28:37.480825+0000","last_deep_scrub":"45'20646","last_deep_scrub_stamp":"2024-09-11T01:28:37.480825+0000","last_clean_scrub_stamp":"2024-09-11T01:28:37.480825+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:46.230705+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17039,"num_write_kb":26880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":112,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:28.953439+0000","last_change":"2024-09-11T01:28:28.953439+0000","last_active":"2024-09-11T01:28:28.953439+0000","last_peered":"2024-09-11T01:28:28.953439+0000","last_clean":"2024-09-11T01:28:28.953439+0000","last_became_active":"2024-09-11T01:14:27.027417+0000","last_became_peered":"2024-09-11T01:14:27.027417+0000","last_unstale":"2024-09-11T01:28:28.953439+0000","last_undegraded":"2024-09-11T01:28:28.953439+0000","last_fullsized":"2024-09-11T01:28:28.953439+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:28:28.953327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:28:28.953327+0000","last_clean_scrub_stamp":"2024-09-11T01:28:28.953327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:50.644608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'20600","reported_seq":20942,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:33.560946+0000","last_change":"2024-09-11T01:28:33.560946+0000","last_active":"2024-09-11T01:28:33.560946+0000","last_peered":"2024-09-11T01:28:33.560946+0000","last_clean":"2024-09-11T01:28:33.560946+0000","last_became_active":"2024-09-11T01:26:47.992146+0000","last_became_peered":"2024-09-11T01:26:47.992146+0000","last_unstale":"2024-09-11T01:28:33.560946+0000","last_undegraded":"2024-09-11T01:28:33.560946+0000","last_fullsized":"2024-09-11T01:28:33.560946+0000","mapping_epoch":54,"log_start":"41'10500","ondisk_log_start":"41'10500","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"45'20600","last_scrub_stamp":"2024-09-11T01:28:33.560846+0000","last_deep_scrub":"45'20600","last_deep_scrub_stamp":"2024-09-11T01:28:33.560846+0000","last_clean_scrub_stamp":"2024-09-11T01:28:33.560846+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:59.411710+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17135,"num_write_kb":27348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4150,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:29.583623+0000","last_change":"2024-09-11T01:28:29.583623+0000","last_active":"2024-09-11T01:28:29.583623+0000","last_peered":"2024-09-11T01:28:29.583623+0000","last_clean":"2024-09-11T01:28:29.583623+0000","last_became_active":"2024-09-11T01:15:20.198333+0000","last_became_peered":"2024-09-11T01:15:20.198333+0000","last_unstale":"2024-09-11T01:28:29.583623+0000","last_undegraded":"2024-09-11T01:28:29.583623+0000","last_fullsized":"2024-09-11T01:28:29.583623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:28:29.583518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:28:29.583518+0000","last_clean_scrub_stamp":"2024-09-11T01:28:29.583518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:46:53.803174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062849,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":131,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:31.553855+0000","last_change":"2024-09-11T01:28:31.553855+0000","last_active":"2024-09-11T01:28:31.553855+0000","last_peered":"2024-09-11T01:28:31.553855+0000","last_clean":"2024-09-11T01:28:31.553855+0000","last_became_active":"2024-09-11T01:15:19.995713+0000","last_became_peered":"2024-09-11T01:15:19.995713+0000","last_unstale":"2024-09-11T01:28:31.553855+0000","last_undegraded":"2024-09-11T01:28:31.553855+0000","last_fullsized":"2024-09-11T01:28:31.553855+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:28:31.553742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:28:31.553742+0000","last_clean_scrub_stamp":"2024-09-11T01:28:31.553742+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:30:04.495144+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00097125399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"63'20140","reported_seq":20429,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:28:32.571913+0000","last_change":"2024-09-11T01:28:32.571913+0000","last_active":"2024-09-11T01:28:32.571913+0000","last_peered":"2024-09-11T01:28:32.571913+0000","last_clean":"2024-09-11T01:28:32.571913+0000","last_became_active":"2024-09-11T01:25:55.123440+0000","last_became_peered":"2024-09-11T01:25:55.123440+0000","last_unstale":"2024-09-11T01:28:32.571913+0000","last_undegraded":"2024-09-11T01:28:32.571913+0000","last_fullsized":"2024-09-11T01:28:32.571913+0000","mapping_epoch":50,"log_start":"41'10100","ondisk_log_start":"41'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"63'20140","last_scrub_stamp":"2024-09-11T01:28:32.571780+0000","last_deep_scrub":"63'20140","last_deep_scrub_stamp":"2024-09-11T01:28:32.571780+0000","last_clean_scrub_stamp":"2024-09-11T01:28:32.571780+0000","objects_scrubbed":2,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:28.527272+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16795,"num_write_kb":26928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4156,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":285231,"num_write_kb":488749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49634,"num_bytes_recovered":36907858,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":52248576,"data_stored":52172703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":347364,"kb_used_data":532,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":94024476,"statfs":{"total":96636764160,"available":96281063424,"internally_reserved":0,"allocated":544768,"data_stored":177817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4434,"internal_metadata":355135150},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":37,"seq":158913790113,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545904,"kb_used_data":544,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825936,"statfs":{"total":96636764160,"available":96077758464,"internally_reserved":0,"allocated":557056,"data_stored":178802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498228,"kb_used_data":51492,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93873612,"statfs":{"total":96636764160,"available":96126578688,"internally_reserved":0,"allocated":52727808,"data_stored":52342328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":457436842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":52199424,"data_stored":52166907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:28:47.007 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T01:28:47.325 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T01:28:47.325 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T01:28:47.325 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T01:28:47.326 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:50.188 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-11T01:28:53.428 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T01:28:53.428 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T01:28:53.429 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:53.499 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T01:28:53.500 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T01:28:53.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:55.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-11T01:28:55.591 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-11T01:28:59.601 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T01:28:59.601 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T01:28:59.602 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T01:28:59.602 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:59.632 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T01:28:59.632 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T01:28:59.633 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T01:28:59.633 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:59.686 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T01:28:59.686 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T01:28:59.687 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:59.740 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T01:28:59.741 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T01:28:59.741 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:28:59.794 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T01:28:59.794 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T01:28:59.794 DEBUG:teuthology.orchestra.run.smithi039:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-09-11T01:28:59.943 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:28:59.943 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T01:29:00.579 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:29:00.580 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T01:29:00.592 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-11T01:29:00.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:29:00.747 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T01:29:00.818 INFO:tasks.ceph:Archiving mon data... 2024-09-11T01:29:00.819 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880/data/mon.a.tgz 2024-09-11T01:29:00.820 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-11T01:29:00.844 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.jznspzNky5 2024-09-11T01:29:00.844 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.jznspzNky5 2024-09-11T01:29:00.979 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.jznspzNky5 2024-09-11T01:29:01.061 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.jznspzNky5 is 311KB 2024-09-11T01:29:01.157 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.jznspzNky5 2024-09-11T01:29:01.173 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880/data/mon.b.tgz 2024-09-11T01:29:01.173 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-11T01:29:01.227 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.mwGiDcyljQ 2024-09-11T01:29:01.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mwGiDcyljQ 2024-09-11T01:29:01.347 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.mwGiDcyljQ 2024-09-11T01:29:01.432 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.mwGiDcyljQ is 314KB 2024-09-11T01:29:01.499 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.mwGiDcyljQ 2024-09-11T01:29:01.514 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880/data/mon.c.tgz 2024-09-11T01:29:01.514 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2024-09-11T01:29:01.568 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.OBXFFxfq6C 2024-09-11T01:29:01.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.OBXFFxfq6C 2024-09-11T01:29:01.688 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.OBXFFxfq6C 2024-09-11T01:29:01.773 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.OBXFFxfq6C is 314KB 2024-09-11T01:29:01.866 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.OBXFFxfq6C 2024-09-11T01:29:01.882 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T01:29:01.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T01:29:02.021 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T01:29:02.021 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T01:29:02.023 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880/remote/smithi039/crash 2024-09-11T01:29:02.024 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T01:29:02.086 INFO:tasks.ceph:Compressing logs... 2024-09-11T01:29:02.086 DEBUG:teuthology.orchestra.run.smithi039:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64797.log 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68241.log 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64797.log.gz 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65049.log 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55129.log 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68241.log.gz 2024-09-11T01:29:02.410 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62360.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65049.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47820.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55129.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68829.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62360.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47820.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44106.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43555.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68829.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37232.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44106.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48282.log 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43555.log.gz 2024-09-11T01:29:02.411 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41248.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37232.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60007.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48282.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70131.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41248.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56391.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60007.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55887.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70131.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68703.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56391.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37982.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55887.log.gz 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47862.log 2024-09-11T01:29:02.412 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68703.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58116.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37982.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48198.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47862.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58116.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.70887.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58116.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55719.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48198.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57567.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70887.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60889.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55719.log.gz 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47232.log 2024-09-11T01:29:02.413 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57567.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65973.log 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60889.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66561.log 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47232.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54877.log 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65973.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69165.log 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66561.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51096.log 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54877.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52860.log/var/log/ceph/ceph-client.admin.69165.log: 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69165.log.gz 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62780.log 2024-09-11T01:29:02.414 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51096.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70509.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52860.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52860.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61478.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62780.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50928.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70509.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51558.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61478.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61478.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37084.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50928.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47988.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51558.log.gz 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68199.log 2024-09-11T01:29:02.415 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37084.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53154.log 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47988.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46098.log 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68199.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56811.log 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53154.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50592.log 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46098.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49164.log 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56811.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46266.log 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50592.log.gz 2024-09-11T01:29:02.416 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50172.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49164.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54036.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46266.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50424.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50172.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64671.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54036.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64545.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50424.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50424.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39556.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64671.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35339.log 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64545.log.gz 2024-09-11T01:29:02.417 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60973.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39556.log.gz 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63158.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35339.log.gz 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46434.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60973.log.gz 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52230.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63158.log.gz 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55677.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46434.log.gz 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61058.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52230.log.gz 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66603.log 2024-09-11T01:29:02.418 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55677.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67737.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61058.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66393.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66603.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43847.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67737.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70173.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66393.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61436.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43847.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52398.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70173.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57525.log 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61436.log.gz 2024-09-11T01:29:02.419 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59419.log 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52398.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69627.log 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57525.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59419.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44587.log 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64208.log 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69627.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47736.log 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44587.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62192.log 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64208.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47736.log.gz 2024-09-11T01:29:02.420 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64629.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64923.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62192.log.gz 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49794.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64629.log.gz 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43884.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64923.log.gz 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67989.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49794.log.gz 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71181.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43884.log.gz 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42883.log 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67989.log.gz 2024-09-11T01:29:02.421 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66225.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71181.log.gz 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42883.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42883.log.gz 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49038.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66225.log.gz 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39914.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36158.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49038.log.gz 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66309.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39914.log.gz 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42337.log 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36158.log.gz 2024-09-11T01:29:02.422 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37612.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66309.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67905.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42337.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52566.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37612.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49500.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67905.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42505.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52566.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57189.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49500.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53574.log 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42505.log.gz 2024-09-11T01:29:02.423 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58368.log 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57189.log.gz 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63914.log 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53574.log.gz 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38352.log 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58368.log.gz 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49752.log 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63914.log.gz 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57231.log 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38352.log.gz 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50718.log 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49752.log.gz 2024-09-11T01:29:02.424 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57231.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47064.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50718.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50718.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41416.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46686.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47064.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44680.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41416.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57315.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46686.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37834.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44680.log.gz 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41668.log 2024-09-11T01:29:02.425 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57315.log.gz 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71307.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37834.log.gz 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46560.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41668.log.gz 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53868.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71307.log.gz 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46560.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.46560.log.gz /var/log/ceph/ceph-client.admin.42253.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57273.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53868.log.gz 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62276.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42253.log.gz 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38056.log 2024-09-11T01:29:02.426 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57273.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62276.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.62276.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41917.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44069.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38056.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56937.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41917.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51978.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56937.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49836.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65469.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51978.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51978.log.gz 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 2024-09-11T01:29:02.427 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49836.log.gz 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65763.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65469.log.gz 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43051.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz --verbose 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.65007.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48324.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65763.log.gz 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38241.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65007.log.gz 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42799.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48324.log.gz 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38241.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38241.log.gz 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35594.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63074.log 2024-09-11T01:29:02.428 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42799.log.gz 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65553.log 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35594.log.gz 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51768.log 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63074.log.gz 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65553.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.65553.log.gz -5 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46728.log 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66519.log 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51768.log.gz 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56601.log 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46728.log.gz 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36157.log 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66519.log.gz 2024-09-11T01:29:02.429 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56601.log.gzgzip 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71055.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.36157.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46812.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36157.log.gz 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58410.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71055.log.gz 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60133.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46812.log.gz 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58410.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58410.log.gz -5 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37649.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.60133.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66939.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60133.log.gz 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46056.log 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37649.log.gz 2024-09-11T01:29:02.430 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31866.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66939.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46056.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56517.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57357.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31866.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38130.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56517.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43921.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57357.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56643.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38130.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41268.log 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43921.log.gz 2024-09-11T01:29:02.431 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36603.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56643.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70677.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41268.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53658.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36603.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61352.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70677.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57651.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53658.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62066.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61352.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37760.log 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57651.log.gz 2024-09-11T01:29:02.432 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70929.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62066.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36862.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37760.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46770.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70929.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37473.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36862.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46770.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37473.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68661.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59545.log 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-09-11T01:29:02.433 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68661.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68661.log.gz 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37121.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46224.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59545.log.gz 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37121.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37121.log.gz 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55382.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62318.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46224.log.gz 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55382.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55382.log.gz -5 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50214.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37343.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62318.log.gz 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47694.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50214.log.gz 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62822.log 2024-09-11T01:29:02.434 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37343.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38463.log 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47694.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57483.log 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62822.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54751.log 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38463.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36390.log 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57483.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65133.log 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54751.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37417.log 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36390.log.gz 2024-09-11T01:29:02.435 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67023.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65133.log.gz 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58662.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37417.log.gz 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35849.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67023.log.gz 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36714.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58662.log.gz 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63704.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35849.log.gz 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51306.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36714.log.gz 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55593.log 2024-09-11T01:29:02.436 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63704.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70257.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48240.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55593.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70257.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69669.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48240.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58873.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69669.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69669.log.gz 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70803.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62234.log 2024-09-11T01:29:02.437 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58873.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61772.log 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70803.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56097.log 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62234.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52482.log 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61772.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58915.log 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56097.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55971.log 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52482.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57864.log 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58915.log.gz 2024-09-11T01:29:02.438 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46014.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55971.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61688.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57864.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62444.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46014.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52314.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61688.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46980.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62444.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65343.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52314.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65301.log 0.0% 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46980.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54709.log 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65343.log.gz 2024-09-11T01:29:02.439 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60511.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65301.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58789.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54709.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69879.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60511.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66435.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58789.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53028.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69879.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64755.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66435.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59755.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53028.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64839.log 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64755.log.gz 2024-09-11T01:29:02.440 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36566.log 0.0% 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59755.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49206.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64839.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47022.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36566.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48450.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49206.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47106.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47022.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63662.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48450.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59713.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47106.log.gz 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-09-11T01:29:02.441 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63662.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67821.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59713.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44291.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54373.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67821.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58536.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44291.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57441.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54373.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59839.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58536.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64461.log 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57441.log.gz 2024-09-11T01:29:02.442 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44513.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59839.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48870.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64461.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56265.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44513.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71391.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48870.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61940.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56265.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39452.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71391.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39639.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61940.log.gz 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61100.log 2024-09-11T01:29:02.443 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39452.log.gz 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39639.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.41183.log -- replaced with /var/log/ceph/ceph-client.admin.39639.log.gz 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.61100.log: /var/log/ceph/ceph-client.admin.36825.log 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61100.log.gz 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39640.log 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41183.log.gz 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49416.log 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36825.log.gz 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39640.log.gzgzip 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56013.log 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63620.log 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49416.log.gz 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51936.log 2024-09-11T01:29:02.444 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56013.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36677.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63620.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51936.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56895.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63998.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36677.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57609.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56895.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51726.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63998.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49584.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57609.log.gz 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62654.log 2024-09-11T01:29:02.445 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51726.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51684.log 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49584.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35468.log 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62654.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55509.log 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51684.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59209.log 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35468.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37908.log 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55509.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59209.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59209.log.gz 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54247.log 2024-09-11T01:29:02.446 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62738.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37908.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54247.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54247.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69963.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62738.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47358.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69963.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37306.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53616.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47358.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70761.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37306.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37306.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47610.log 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53616.log.gz 2024-09-11T01:29:02.447 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43219.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70761.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37945.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47610.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69711.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43219.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53826.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37945.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49626.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69711.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70047.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53826.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67569.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49626.log.gz 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-09-11T01:29:02.448 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70047.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34922.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67569.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49374.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69921.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34922.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58999.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49374.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47400.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69921.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49332.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58999.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39788.log 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47400.log.gz 2024-09-11T01:29:02.449 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71223.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49332.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68745.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39788.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61856.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71223.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51516.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68745.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61856.log.gzgzip 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37515.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51138.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51516.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47526.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37515.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51810.log 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51138.log.gz 2024-09-11T01:29:02.450 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70341.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47526.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35340.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51810.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59671.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70341.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50760.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35340.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58704.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59671.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50466.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50760.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36306.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58704.log.gz 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-09-11T01:29:02.451 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50466.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36306.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47652.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42421.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61015.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47652.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42421.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36348.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61015.log.gz 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45762.log 2024-09-11T01:29:02.452 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66813.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36348.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66981.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45762.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47946.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66813.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52524.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66981.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49290.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47946.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52734.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52524.log.gz 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49290.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.49290.log.gz --verbose 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.53910.log 2024-09-11T01:29:02.453 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59587.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52734.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37575.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53910.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.59587.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50298.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59587.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48912.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37575.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42169.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50298.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48618.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48912.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53490.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42169.log.gz 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46896.log 2024-09-11T01:29:02.454 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48618.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64040.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53490.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42841.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46896.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41604.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64040.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67191.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42841.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69375.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41604.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46392.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67191.log.gz 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.69375.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68577.log 2024-09-11T01:29:02.455 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69375.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67359.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46392.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56475.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68577.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66897.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67359.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53070.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56475.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42631.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66897.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68031.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53070.log.gz 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65889.log 2024-09-11T01:29:02.456 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42631.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55045.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68031.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35762.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65889.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39830.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55045.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47904.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35762.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62696.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39830.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68787.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47904.log.gz 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37047.log 2024-09-11T01:29:02.457 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62696.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37269.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68787.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68787.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37047.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54541.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37269.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58242.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69459.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54541.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48576.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58242.log.gz 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59041.log 2024-09-11T01:29:02.458 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69459.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55213.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48576.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32045.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59041.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43639.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55213.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43093.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32045.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67233.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43639.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69081.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43093.log.gz 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63410.log 2024-09-11T01:29:02.459 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67233.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58074.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69081.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46308.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63410.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69501.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58074.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41458.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46308.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68913.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69501.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65805.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41458.log.gz 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36529.log 2024-09-11T01:29:02.460 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68913.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65805.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50004.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36529.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36529.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67317.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63032.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50004.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53532.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67317.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63032.log.gz 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58284.log 2024-09-11T01:29:02.461 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53532.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50088.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58284.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38167.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48492.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50088.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59461.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38167.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70299.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48492.log.gz 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68367.log 2024-09-11T01:29:02.462 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59461.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64503.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70299.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71349.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68367.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52146.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64503.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57693.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71349.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61898.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52146.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62024.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57693.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61898.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61898.log.gz 2024-09-11T01:29:02.463 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62570.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62024.log.gz 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60175.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69795.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62570.log.gz 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55297.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60175.log.gz 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43009.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69795.log.gz 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37010.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55297.log.gz 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46350.log 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43009.log.gz 2024-09-11T01:29:02.464 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60847.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37010.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47568.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46350.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48156.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60847.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45972.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47568.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56181.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48156.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32116.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45972.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51012.log 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56181.log.gz 2024-09-11T01:29:02.465 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60049.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.32116.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32116.log.gz 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55425.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51012.log.gz 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66183.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60049.log.gz 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56769.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55425.log.gz 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54163.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66183.log.gz 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52902.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56769.log.gz 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-11T01:29:02.466 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54163.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48954.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52902.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65175.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56685.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48954.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52944.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65175.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35807.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56685.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35989.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52944.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60637.log 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35807.log.gz 2024-09-11T01:29:02.467 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45804.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35989.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57738.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60637.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35990.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45804.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57738.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.57738.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.43958.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56727.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35990.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43958.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.43958.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37686.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38019.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56727.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37538.log 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37686.log.gz 2024-09-11T01:29:02.468 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38019.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60217.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37195.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr: 96.2% 0.0% -- replaced with /var/log/ceph/ceph.audit.log.gz -- replaced with /var/log/ceph/ceph-client.admin.37538.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60217.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60217.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63284.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37195.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53238.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35006.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63284.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53238.log: /var/log/ceph/ceph-client.admin.44328.log 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53238.log.gz 2024-09-11T01:29:02.469 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42085.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35006.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54331.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44328.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56349.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42085.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67401.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54331.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50340.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56349.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61226.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67401.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46518.log 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50340.log.gz 2024-09-11T01:29:02.470 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55635.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61226.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43387.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46518.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55635.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43387.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47274.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47190.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38315.log 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47274.log.gz 2024-09-11T01:29:02.471 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54078.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47190.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68157.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38315.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47148.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54078.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54625.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68157.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43307.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47148.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46182.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54625.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62906.log 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43307.log.gz 2024-09-11T01:29:02.472 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53742.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46182.log.gz 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51054.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62906.log.gz 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38426.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53742.log.gz 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64419.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51054.log.gz 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69585.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38426.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38426.log.gz 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66141.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64419.log.gz 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2024-09-11T01:29:02.473 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69585.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.66141.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31906.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66141.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31906.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68283.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56307.log: -- /var/log/ceph/ceph-client.admin.68997.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51852.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68283.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55339.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68997.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36973.log 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51852.log.gz 2024-09-11T01:29:02.474 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49878.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55339.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58158.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36973.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39555.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49878.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45720.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58158.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53448.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39555.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68871.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45720.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53448.log.gz 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.68871.log: /var/log/ceph/ceph-client.admin.52608.log 2024-09-11T01:29:02.475 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68871.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63368.log 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36222.log 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52608.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57105.log 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63368.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58957.log 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36222.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67695.log 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57105.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42001.log 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58957.log.gz 2024-09-11T01:29:02.476 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.67695.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.68073.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67695.log.gz 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42001.log.gz 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52104.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68073.log.gz 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50802.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52104.log: /var/log/ceph/ceph-client.admin.64124.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52104.log.gz 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62990.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50802.log.gz 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53700.log 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64124.log.gz 2024-09-11T01:29:02.477 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34964.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62990.log.gz 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53700.log.gz 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35212.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34964.log.gz 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62864.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35212.log.gz 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44476.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62864.log.gz 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60469.log 2024-09-11T01:29:02.478 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44476.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51474.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60469.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38500.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57822.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51474.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54961.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38500.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49668.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57822.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59125.log 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54961.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54961.log.gz 2024-09-11T01:29:02.479 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65637.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49668.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37797.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59125.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43995.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65637.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61814.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37797.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68325.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43995.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59965.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61814.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61646.log 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68325.log.gz 2024-09-11T01:29:02.480 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59965.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59503.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61646.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48408.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62528.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59503.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59503.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48408.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57990.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62528.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61394.log 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57990.log.gz 2024-09-11T01:29:02.481 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53784.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41833.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61394.log.gz 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49920.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53784.log.gz 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53952.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41833.log.gz 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53406.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49920.log.gz 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41015.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53952.log.gz 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63830.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53406.log.gz 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-09-11T01:29:02.482 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41015.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44032.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63830.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31569.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31747.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44032.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31569.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27150.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31569.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43429.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31747.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.tmp-client.admin.27150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56139.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27150.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57021.log 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43429.log.gz 2024-09-11T01:29:02.483 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.56139.log: -- /var/log/ceph/ceph-client.admin.55003.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56139.log.gz 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67149.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57021.log.gz 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55003.log.gz 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67149.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67149.log.gz 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35914.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53322.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70467.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35914.log.gz 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34826.log 2024-09-11T01:29:02.484 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53322.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70719.log 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70467.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68535.log 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34826.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41752.log 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70719.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64292.log 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68535.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36936.log 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41752.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35338.log 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64292.log.gz 2024-09-11T01:29:02.485 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68493.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36936.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46938.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35338.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70089.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68493.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35213.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46938.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46140.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70089.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46602.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35213.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63536.log 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46140.log.gz 2024-09-11T01:29:02.486 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39956.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46602.log.gz 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63536.log.gz 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67443.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39956.log.gz 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59335.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48072.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67443.log.gz 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65259.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59335.log.gz 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66645.log 2024-09-11T01:29:02.487 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48072.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49248.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65259.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68451.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66645.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71013.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49248.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41520.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68451.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34736.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71013.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41959.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41520.log.gz 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50886.log 2024-09-11T01:29:02.488 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34736.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53196.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41959.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57147.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50886.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67863.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53196.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41875.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57147.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48702.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67863.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70425.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41875.log.gz 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49458.log 2024-09-11T01:29:02.489 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48702.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70425.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.70425.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64881.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35088.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49458.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36074.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64881.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64250.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35088.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50046.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36074.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56979.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64250.log.gz 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52818.log 2024-09-11T01:29:02.490 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50046.log.gz 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55803.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56979.log.gz 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58032.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52818.log.gz 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55803.log.gz 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43662.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58032.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58032.log.gz -5 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50256.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43662.log.gz 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50676.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50256.log.gz 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36492.log 2024-09-11T01:29:02.491 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50676.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.69039.log 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50676.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60301.log 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36492.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69039.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63956.log 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52020.log 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60301.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63956.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36264.log 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52020.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52020.log.gz 2024-09-11T01:29:02.492 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41584.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36264.log.gz 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42757.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41584.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.41584.log.gz 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.69207.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42757.log.gz 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41099.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69207.log.gz 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44143.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57906.log 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41099.log.gz 2024-09-11T01:29:02.493 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60427.log 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44143.log.gz 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66687.log 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57906.log.gz 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69333.log 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60427.log.gz 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52062.log 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66687.log.gz 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43323.log 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69333.log.gz 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51432.log 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52062.log.gz 2024-09-11T01:29:02.494 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55467.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43323.log.gz 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45930.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51432.log.gz 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55467.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.55467.log.gz --verbose 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.63326.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35510.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45930.log.gz 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.63326.log: /var/log/ceph/ceph-client.admin.64965.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63326.log.gz 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55255.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35510.log.gz 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67275.log 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64965.log.gz 2024-09-11T01:29:02.495 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69543.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55255.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67275.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64587.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68955.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69543.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64335.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64587.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60763.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68955.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64335.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64335.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65385.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50130.log 2024-09-11T01:29:02.496 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60763.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63788.log 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65385.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42211.log 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50130.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63242.log 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63788.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65217.log 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42211.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60721.log 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63242.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63578.log 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65217.log.gz 2024-09-11T01:29:02.497 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52986.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60721.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67527.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63578.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42379.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52986.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52650.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67527.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45846.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42379.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52650.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52650.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62612.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45846.log.gz 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62486.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60385.log 2024-09-11T01:29:02.498 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62612.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44439.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62486.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54499.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60385.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64377.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44439.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42463.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54499.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58578.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64377.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57399.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42463.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56559.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58578.log.gz 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70593.log 2024-09-11T01:29:02.499 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57399.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63116.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56559.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50970.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70593.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31946.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63116.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70635.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50970.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.31946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62402.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31946.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.70635.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.58494.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70635.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62402.log.gz 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59797.log 2024-09-11T01:29:02.500 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58494.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69249.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56433.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59797.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39494.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69249.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56433.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35891.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39494.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40954.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66729.log 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35891.log.gz 2024-09-11T01:29:02.501 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.40954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71139.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40954.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41688.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66729.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67611.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71139.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58620.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41688.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67611.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38389.log -- replaced with /var/log/ceph/ceph-client.admin.67611.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40912.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58620.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69837.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38389.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38389.log.gz 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70845.log 2024-09-11T01:29:02.502 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.40912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40912.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65679.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69837.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59083.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70845.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49962.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65679.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67947.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59083.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67485.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49962.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49962.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39704.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67947.log.gz 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56223.log 2024-09-11T01:29:02.503 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67485.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41775.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39704.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34697.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56223.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44550.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67653.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34697.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36640.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44550.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52440.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67653.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61730.log 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36640.log.gz 2024-09-11T01:29:02.504 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65721.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52440.log.gz 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35426.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61730.log.gz 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55845.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65721.log.gz 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55929.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35426.log.gz 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48030.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55845.log.gz 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59251.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55929.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55929.log.gz 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61142.log 2024-09-11T01:29:02.505 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48030.log.gz 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59251.log.gz 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45678.log 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61142.log.gz 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37158.log 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.45678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59923.log 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45678.log.gz 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52356.log 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37158.log.gz 2024-09-11T01:29:02.506 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54289.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59923.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46854.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52356.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52356.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36455.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54289.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65511.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46854.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35086.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36455.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54415.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65511.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70551.log 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35086.log.gz 2024-09-11T01:29:02.507 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48366.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54415.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46644.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70551.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64166.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48366.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51642.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46644.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42295.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64166.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64166.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59377.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51642.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62948.log 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42295.log.gz 2024-09-11T01:29:02.508 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51894.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59377.log.gz 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62948.log.gz 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60679.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68619.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51894.log.gz 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60679.log.gz 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67065.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68619.log.gz 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63200.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56853.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67065.log.gz 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53994.log 2024-09-11T01:29:02.509 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63200.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52776.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56853.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64082.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53994.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42925.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52776.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69123.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64082.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66771.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42925.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36788.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69123.log.gz 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60931.log 2024-09-11T01:29:02.510 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66771.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38093.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36788.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44624.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60931.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61982.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38093.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71431.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44624.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35214.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61982.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67779.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71431.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37871.log 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35214.log.gz 2024-09-11T01:29:02.511 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60259.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67779.log.gz 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59881.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37871.log.gz 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54919.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60259.log.gz 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5/var/log/ceph/ceph-client.admin.59881.log: --verbose -- /var/log/ceph/ceph-client.admin.47484.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59881.log.gz 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58831.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54919.log.gz 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66477.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47484.log.gz 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53112.log 2024-09-11T01:29:02.512 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58831.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57780.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66477.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44402.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53112.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57948.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57780.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42127.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44402.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66015.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57948.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54205.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42127.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42127.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35552.log 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66015.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66015.log.gz 2024-09-11T01:29:02.513 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65595.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54205.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61310.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35552.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65595.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36432.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53364.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61310.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43513.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36432.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61184.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53364.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43513.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.43513.log.gz -- 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: /var/log/ceph/ceph-client.admin.39872.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36751.log 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61184.log.gz 2024-09-11T01:29:02.514 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49710.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39872.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67107.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36751.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61604.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.49710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49710.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.67107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37723.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67107.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37380.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61604.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51600.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37723.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.37380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37380.log.gz 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69417.log 2024-09-11T01:29:02.515 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51600.log.gz 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61520.log 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54667.log 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69417.log.gz 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62150.log 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61520.log.gz 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54793.log 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54667.log.gz 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34880.log 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62150.log.gz 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54793.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.54793.log.gz --verbose 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr: -- /var/log/ceph/ceph-client.admin.61562.log 2024-09-11T01:29:02.516 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70383.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34880.log.gz 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65091.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61562.log.gz 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57063.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70383.log.gz 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61268.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65091.log.gz 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68115.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.57063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57063.log.gz 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.61268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63452.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61268.log.gz 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65427.log 2024-09-11T01:29:02.517 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68115.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68409.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63452.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63452.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.65427.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69753.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65427.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.68409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66267.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68409.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44365.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69753.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47442.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66267.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35678.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44365.log.gz 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62108.log 2024-09-11T01:29:02.518 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47442.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35087.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35678.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.62108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62108.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63872.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35087.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35087.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71097.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63872.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.44254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-09-11T01:29:02.519 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71097.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55087.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38278.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60805.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55087.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.38278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52188.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38278.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52692.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60805.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60805.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52272.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52188.log.gz 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51390.log 2024-09-11T01:29:02.520 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52692.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63746.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.52272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52272.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50382.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51390.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70971.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63746.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51222.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50382.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59629.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70971.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53280.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51222.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42547.log 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59629.log.gz 2024-09-11T01:29:02.521 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.53280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47778.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53280.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42715.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42547.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50508.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47778.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.42715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64713.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42715.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43239.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50508.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47316.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.64713.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64713.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54583.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43239.log.gz 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.47316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70215.log 2024-09-11T01:29:02.522 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47316.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58200.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54583.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63494.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70215.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58747.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr: 28.2% -- replaced with /var/log/ceph/ceph-client.admin.58200.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.63494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63494.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58326.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58747.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.50844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65931.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43471.log 2024-09-11T01:29:02.523 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58326.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59167.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65931.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48744.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43471.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59167.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59167.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:gzip/var/log/ceph/ceph-client.admin.48744.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.66057.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48744.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54120.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71265.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66057.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70005.log 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54120.log.gz 2024-09-11T01:29:02.524 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43154.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.71265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71265.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.70005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41332.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70005.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.43154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59293.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43154.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.41332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69291.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41332.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.59293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34792.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59293.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55551.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.69291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69291.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55761.log 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.34792.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34792.log.gz 2024-09-11T01:29:02.525 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56055.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55551.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66099.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.55761.log: 95.7% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41775.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55761.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.56055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60553.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56055.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46476.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66099.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60091.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60553.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66855.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.46476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46476.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65847.log 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60091.log.gz 2024-09-11T01:29:02.526 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51348.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66855.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66351.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.65847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65847.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.51348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58452.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51348.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.66351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39724.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66351.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.58452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36899.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58452.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.39724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60343.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39724.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.36899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36899.log.gz 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.60343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35720.log 2024-09-11T01:29:02.527 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60343.log.gz 2024-09-11T01:29:02.528 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.54457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48828.log 2024-09-11T01:29:02.528 INFO:teuthology.orchestra.run.smithi039.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-09-11T01:29:02.528 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.35720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35720.log.gz 2024-09-11T01:29:02.528 INFO:teuthology.orchestra.run.smithi039.stderr:/var/log/ceph/ceph-client.admin.48828.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48828.log.gz 2024-09-11T01:29:03.762 INFO:teuthology.orchestra.run.smithi039.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T01:29:04.199 INFO:teuthology.orchestra.run.smithi039.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T01:29:04.393 INFO:teuthology.orchestra.run.smithi039.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T01:29:05.920 INFO:teuthology.orchestra.run.smithi039.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T01:37:57.257 INFO:teuthology.orchestra.run.smithi039.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T01:42:29.717 INFO:teuthology.orchestra.run.smithi039.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T01:43:58.211 INFO:teuthology.orchestra.run.smithi039.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T01:43:58.213 INFO:teuthology.orchestra.run.smithi039.stderr: 2024-09-11T01:43:58.213 INFO:teuthology.orchestra.run.smithi039.stderr:real 14m56.071s 2024-09-11T01:43:58.213 INFO:teuthology.orchestra.run.smithi039.stderr:user 6m15.120s 2024-09-11T01:43:58.213 INFO:teuthology.orchestra.run.smithi039.stderr:sys 0m15.823s 2024-09-11T01:43:58.214 INFO:tasks.ceph:Archiving logs... 2024-09-11T01:43:58.399 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880/remote/smithi039/log 2024-09-11T01:43:58.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T01:45:03.217 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T01:45:03.227 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T01:45:03.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T01:45:03.322 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T01:45:03.330 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T01:45:03.330 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T01:45:03.348 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: ntpq: command not found 2024-09-11T01:45:03.396 INFO:teuthology.orchestra.run.smithi039.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-11T01:45:03.396 INFO:teuthology.orchestra.run.smithi039.stdout:=============================================================================== 2024-09-11T01:45:03.396 INFO:teuthology.orchestra.run.smithi039.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -611us[ -609us] +/- 62ms 2024-09-11T01:45:03.396 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 63 +1105us[+1106us] +/- 59ms 2024-09-11T01:45:03.396 INFO:teuthology.orchestra.run.smithi039.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 194 -842us[ -811us] +/- 76ms 2024-09-11T01:45:03.396 INFO:teuthology.orchestra.run.smithi039.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-11T01:45:03.397 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T01:45:03.405 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T01:45:03.406 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T01:45:03.420 DEBUG:teuthology.orchestra.run.smithi039:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-11T01:45:03.558 DEBUG:teuthology.orchestra.run.smithi039:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-11T01:45:03.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:45:03.589 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T01:45:03.598 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T01:45:03.607 INFO:teuthology.task.internal:Duration was 2303.572804 seconds 2024-09-11T01:45:03.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T01:45:03.626 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T01:45:03.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T01:45:03.689 INFO:teuthology.orchestra.run.smithi039.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-11T01:45:03.796 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T01:45:03.796 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2024-09-11T01:45:03.796 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T01:45:03.817 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T01:45:03.818 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T01:45:03.886 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T01:45:03.886 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T01:45:04.070 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T01:45:04.080 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T01:45:04.081 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T01:45:04.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T01:45:04.123 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T01:45:04.187 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2024-09-11T01:45:04.206 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T01:45:04.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:45:04.261 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T01:45:04.270 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T01:45:04.271 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899880/remote/smithi039 2024-09-11T01:45:04.271 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T01:45:04.335 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T01:45:04.336 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T01:45:04.387 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T01:45:04.396 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T01:45:04.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T01:45:04.460 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T01:45:04.460 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T01:45:04.476 INFO:teuthology.orchestra.run.smithi039.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 01:45 /home/ubuntu/cephtest 2024-09-11T01:45:04.477 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T01:45:04.514 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T01:45:04.541 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/few msgr/async-v2only objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-lz4} rados supported-random-distro$/{centos_latest}} duration: 2303.5728044509888 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-11T01:45:04.542 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T01:45:04.640 INFO:teuthology.run:pass